Understanding Language Server Protocol

Every developer has their favorite IDE, it may be VS Code, something from the JetBrains family, Zed or any IDE. But have you ever wondered how your IDE gives you features like autocompletion, go-to-definition, and hover info? Not just for one language, but for so many.

If you have ever tried to figure out how that works, you would have probably come across the term Language Server Protocol (LSP).

This blog will help you understand what LSP is, how it works, and why it plays a key role in making editors smarter.

We will walk through

1. How IDEs worked before LSP

2. The challenges of the old approach

3. What is LSP

4. How LSP solved the old problems

5. How LSP works

6. Language Server Capabilities

7. LSP supported Language Servers

IDEs before LSP

Back in the day, writing code was totally a different experience. We used to type entire program in a text editor, compile them manually, fix the errors, recompile, and sometimes endlessly. It was frustrating, but looking back, those were the golden days of programming.

As time went on, we moved from simple text editors to proper code editors what we now call IDEs. In the early days, most IDEs were built to support just one programming language. But as more languages became popular, editors had to evolve. So, they started supporting multiple languages using extensions or plugins like in the diagram below.

These extensions acted as a bridge between the editor and the language tool. The language tool understands your code and provides language-specific features like autocompletion, go-to-definition, hover info, and error highlighting. To support rich code editing features, editors had to tightly integrate with every language tooling.

The challenges of the old approach

The above diagram (before LSP Diagram) shows two things clear.

1. The editor has to build custom integrations for every language.

2. Language provider has to build separate plugin for the same language just to support different editors.

Adding features like autocomplete, go-to-definition, or hover info for a language is not easy. And to make it worse, language providers had to repeat the same work for every editor, since each editor came with its own APIs.

And there were deeper challenges too:

• Different languages, different runtimes:
  Language Tools are often written in different languages than the editor. For example, the language tool might be in Python or Go, but VS Code runs on Node.js. So, getting them to work together was not always smooth.

• Performance impact:
  Language Tools do a lot of heavy work, like parsing code, analyzing syntax, and finding definitions. Running all this inside the editor could slow things down.

• Too much effort to integrate:

  Like we saw earlier, editors had to build separate support for each language tool, and language providers had to write a plugin for every editor, even if it was for the same language.

To address all these issues LSP was introduced.

What is LSP

LSP is a specification (a set of rules) or protocol that defines how editors and language servers talk.

LSP has two main components.

Language Server: A background process that understands your code and provides language specific features.

Language Server is not something new. Before LSP, it was known as a language tool, which was tightly integrated into the editor or IDE. What’s different now is that the language server runs as a separate process and talks to the editor using the LSP.

Language Client: A kind of plugin or extension in an editor that talks to the Language Server, get the features and shows those smart features to you.

Now any LSP compliant server will work with any LSP compliant editor.

How LSP solved old problems

We have seen the high-level diagram above. Now, here's a detailed diagram that shows how one Python language server can work with multiple editors.

We have seen the problems that existed before LSP. Now let us see how LSP makes things easier.

Problem: Different languages, different runtimes

Solution:

The editor and language server are now separate processes. They just exchange messages using LSP (over JSON-RPC), so it doesn’t matter what language either one is written in. As long as both speak the protocol, they can work together.

Problem: Performance

Solution:

Heavy tasks like parsing files and analyzing syntax, stay with the language server and runs away from the editor. So, the editor stays light and responsive. It is a clean separation; both do what they’re good at.

Problem: M × N integrations

Solution: M+N

Now, each editor just needs to have one LSP client, and each language needs to implement one Language Server that follows the LSP standard.
That same Language Server can then be reused across any editor that supports LSP, no extra work needed.

The whole idea behind the Language Server Protocol is to save language developers from the headache of writing separate implementations for every editor.

How LSP works

Okay, we now know that LSP is the standard way editors and language servers talk. But how exactly does that “talking” happen?

Let’s break it down.

LSP is built on top of something called JSON-RPC, it’s a simple way for two programs to send messages back and forth using plain JSON. Here’s a quick diagram showing how the IDE and Language Server exchange messages.

• User opens a file → didOpen: editor tells the server a file is opened

• User edits the file → didChange: editor sends updates as the user types

• Server sends back errors or warnings → publishDiagnostics: server shares issues it found

• User triggers “Go to Definition” → definition: editor asks for symbol location → server responds

• User closes the file → didClose: editor notifies the server that the file was closed

Every time you open a file, edit code, or do something, a carefully choreographed exchange of messages occurs between editor and language server.

To view these message exchange, live in VS Code, you can enable trace logging. Here’s how to do it for the Python Language Server. Make sure the Python extension is installed in VS Code.

• Press Ctrl + Shift + P,

• Type and select, 'Preferences: Open Settings (JSON)',

• Add this line to the settings.json

"python.analysis.logLevel": "Trace"

• Now go to View → Output, then select “Python Language Server” from the dropdown. Start seeing the message exchange when you do something in any python file.

Here’s how the message exchange between the IDE and Language Server looks in action - captured from VS Code.

Below is one of the many messages from the video above. Just to show a clear example of how the LSP client notifies the language server when something changes in the file.

The notification is called textDocument/didChange, and it includes details like:

• Which file was changed ("file:///C:/Users/.../processor.py").

• What version of the file it’s on ("version": 51).

• Where in the file, the change happened (line and character range).

• The actual new text that replaced the old content.

Here are a few more message exchanges that happen when you open or close a file.

'textDocument/didClose' - Client notifies Server that the file has closed.

'textDocument/didOpen'-Client notifies Server that the file has opened

Language Features:

Now let’s look at all the language features that a Language Server can implement and support through LSP.

Language Server Capabilities:

A Language Server does not have to implement all of the above features. It simply tells the editor what features it supports and shares the capabilities during the initial handshake.

At the start of the connection, the client sends an initialization request with its own capabilities. The server then responds with the features it supports. This is called static capability registration.

Below screenshot shows the initialize request from client to server.

The following is the response message sent by the language server in reply to the initialization request. In this response, the server declares its capabilities to the language client.

LSP supported Language Servers:

Language Servers are already available for most popular languages. You can find the full list on this page. If the language server you need is not listed, you can contribute by raising a pull request on GitHub, as explained in the page.

Few for sample,

Go → Go Language Server

JavaScript and Typescript → JavaScript and TypeScript Language Server

PHP → PHP Language Server

Python → Python Language Server

Final Thoughts:

I hope this blog helped you understand what Language Server Protocol is, why it was created, and how it makes your editor smart with features like autocomplete, go-to-definition, and more. LSP is truly a win-win for language providers and editors.

Try exploring some language servers or setting one up in your own editor to see it in action.

Got something cool to share about LSP or language servers? Drop it in the comments. let’s learn together!

Happy Coding!