zed-industries/zed
1
1
Fork 0
mirror of https://github.com/zed-industries/zed.git synced 2024-11-08 07:35:01 +03:00
Code Issues Packages Projects Releases Wiki Activity
49371b44cb
zed/crates
History
Kyle Kelley 49371b44cb
Semantic Index (#10329) 
This introduces semantic indexing in Zed based on chunking text from
files in the developer's workspace and creating vector embeddings using
an embedding model. As part of this, we've created an embeddings
provider trait that allows us to work with OpenAI, a local Ollama model,
or a Zed hosted embedding.

The semantic index is built by breaking down text for known
(programming) languages into manageable chunks that are smaller than the
max token size. Each chunk is then fed to a language model to create a
high dimensional vector which is then normalized to a unit vector to
allow fast comparison with other vectors with a simple dot product.
Alongside the vector, we store the path of the file and the range within
the document where the vector was sourced from.

Zed will soon grok contextual similarity across different text snippets,
allowing for natural language search beyond keyword matching. This is
being put together both for human-based search as well as providing
results to Large Language Models to allow them to refine how they help
developers.

Remaining todo:

* [x] Change `provider` to `model` within the zed hosted embeddings
database (as its currently a combo of the provider and the model in one
name)


Release Notes:

- N/A

---------

Co-authored-by: Nathan Sobo <nathan@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Conrad Irwin <conrad@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Antonio <antonio@zed.dev>
2024-04-12 11:40:59 -06:00
..
activity_indicator Add auto-update system for extensions (#9890) 2024-03-28 15:41:22 -04:00
anthropic Check license generation for every PR to avoid license-less crate additions (#10033) 2024-04-01 12:16:16 +03:00
assets Move Clippy configuration to the workspace level (#8891) 2024-03-05 12:01:17 -05:00
assistant Remove if-not-else patterns (#10402) 2024-04-11 03:48:06 -04:00
audio Introduce InlineCompletionProvider (#9777) 2024-03-26 13:28:06 +01:00
auto_update Refactor workspace notifications to use explicit NotificationId type (#10342) 2024-04-10 17:21:23 -04:00
breadcrumbs Remove line breaks when displaying file names in the project panel (#10231) 2024-04-10 08:30:13 -06:00
call WIP: remoting (#10085) 2024-04-11 15:36:35 -06:00
channel Semantic Index (#10329) 2024-04-12 11:40:59 -06:00
cli Windows: Enable clippy deny warnings (#9920) 2024-03-28 11:55:35 -04:00
client WIP: remoting (#10085) 2024-04-11 15:36:35 -06:00
clock Move Clippy configuration to the workspace level (#8891) 2024-03-05 12:01:17 -05:00
collab Semantic Index (#10329) 2024-04-12 11:40:59 -06:00
collab_ui chat panel: Add timestamp in tooltip to edited message (#10444) 2024-04-12 14:26:41 +02:00
collections gpui: Add SVG rendering to img element and generic asset cache (#9931) 2024-03-29 17:09:49 -07:00
color Fix hex_to_hsla inside the color crate (#9412) 2024-03-15 16:58:46 -04:00
command_palette Encapsulate CommandPaletteFilter and CommandPaletteInterceptor (#9402) 2024-03-15 12:37:25 -04:00
command_palette_hooks Introduce InlineCompletionProvider (#9777) 2024-03-26 13:28:06 +01:00
copilot Revert "Revert "language: Remove buffer fingerprinting (#9007)"" (#9671) 2024-04-12 18:40:35 +02:00
copilot_ui Refactor workspace notifications to use explicit NotificationId type (#10342) 2024-04-10 17:21:23 -04:00
db Fix issues with extension API that come up when moving Svelte into an extension (#9611) 2024-03-22 17:29:06 -07:00
diagnostics Add preview tabs (#9125) 2024-04-11 23:09:12 +02:00
editor Semantic Index (#10329) 2024-04-12 11:40:59 -06:00
extension Extract Terraform extension (#10479) 2024-04-12 11:49:49 -04:00
extension_api Improve extension API documentation (#10322) 2024-04-09 10:38:29 -04:00
extension_cli Add git blame (#8889) 2024-03-28 18:32:11 +01:00
extensions_ui Extract Terraform extension (#10479) 2024-04-12 11:49:49 -04:00
feature_flags WIP: remoting (#10085) 2024-04-11 15:36:35 -06:00
feedback Refactor workspace notifications to use explicit NotificationId type (#10342) 2024-04-10 17:21:23 -04:00
file_finder Add preview tabs (#9125) 2024-04-11 23:09:12 +02:00
file_icons tasks: Use icons instead of secondary text in a modal (#10264) 2024-04-08 11:41:54 +02:00
fs Fix git blame not working correctly with submodules (#10114) 2024-04-03 13:49:12 +02:00
fsevent Add rs-notify implementation of fs::watch (#9040) 2024-03-08 22:18:44 -08:00
fuzzy Move Clippy configuration to the workspace level (#8891) 2024-03-05 12:01:17 -05:00
git Add git blame error reporting with notification (#10408) 2024-04-12 07:20:34 +02:00
go_to_line Line numbers short mode (#10354) 2024-04-10 12:08:07 +02:00
google_ai Fix licensing errors 2024-03-20 15:52:02 +01:00
gpui Semantic Index (#10329) 2024-04-12 11:40:59 -06:00
gpui_macros gpui-macros: Hide autogenerated action types/functions (#10417) 2024-04-11 15:38:47 +02:00
headless WIP: remoting (#10085) 2024-04-11 15:36:35 -06:00
image_viewer Add preview tabs (#9125) 2024-04-11 23:09:12 +02:00
install_cli Fix flickering (#9012) 2024-03-11 10:45:57 +01:00
journal Add the ability for extensions to provide language settings (#10296) 2024-04-08 19:17:12 -04:00
language Semantic Index (#10329) 2024-04-12 11:40:59 -06:00
language_selector Move Clippy configuration to the workspace level (#8891) 2024-03-05 12:01:17 -05:00
language_tools Add preview tabs (#9125) 2024-04-11 23:09:12 +02:00
languages Extract Terraform extension (#10479) 2024-04-12 11:49:49 -04:00
live_kit_client Add a schema to extensions, to prevent installing extensions on too old of a Zed version (#9599) 2024-03-20 17:33:26 -04:00
live_kit_server Merge prost dependecies (#9522) 2024-03-18 20:33:20 -06:00
lsp Show status of LSP actions (#9818) 2024-04-06 19:48:11 -07:00
markdown_preview Add preview tabs (#9125) 2024-04-11 23:09:12 +02:00
media Revert "Revert "Revert dependency updates in #9836 (#10089)"" 2024-04-02 13:12:38 -06:00
menu Fix the linux keymap (#9829) 2024-03-26 16:10:09 -07:00
multi_buffer vim: Don’t allow edits in the read-only state (#10404) 2024-04-11 18:19:49 -06:00
node_runtime Allow extensions to define more of the methods in the LspAdapter trait (#9554) 2024-03-20 12:47:04 -07:00
notifications Fix mention notifications are not updated after message change and not removed after a message is deleted (#9847) 2024-04-02 20:40:00 -06:00
open_ai Semantic Index (#10329) 2024-04-12 11:40:59 -06:00
outline Restore --all-targets for clippy (#9346) 2024-03-14 12:39:45 -04:00
picker tab_switcher: Add tab close buttons (#9968) 2024-04-03 17:28:51 +02:00
prettier Fix prettier-plugin-organize-imports plugin removes used imports (#9598) 2024-03-21 08:23:15 +01:00
project Semantic Index (#10329) 2024-04-12 11:40:59 -06:00
project_panel Organize Project Panel context menus (#10456) 2024-04-11 22:52:49 -04:00
project_symbols Move Clippy configuration to the workspace level (#8891) 2024-03-05 12:01:17 -05:00
quick_action_bar Improve Find/Replace shortcuts (#10297) 2024-04-08 22:07:59 -07:00
recent_projects Remove if-not-else patterns (#10402) 2024-04-11 03:48:06 -04:00
refineable Move Clippy configuration to the workspace level (#8891) 2024-03-05 12:01:17 -05:00
release_channel Move Clippy configuration to the workspace level (#8891) 2024-03-05 12:01:17 -05:00
rich_text chat panel: Add timestamp in tooltip to edited message (#10444) 2024-04-12 14:26:41 +02:00
rope Revert "Revert "language: Remove buffer fingerprinting (#9007)"" (#9671) 2024-04-12 18:40:35 +02:00
rpc Semantic Index (#10329) 2024-04-12 11:40:59 -06:00
search Fix overlap (#10474) 2024-04-12 08:25:09 -06:00
semantic_index Semantic Index (#10329) 2024-04-12 11:40:59 -06:00
semantic_version Extract SemanticVersion into its own crate (#9956) 2024-03-29 12:11:57 -04:00
settings Remove if-not-else patterns (#10402) 2024-04-11 03:48:06 -04:00
snippet Move Clippy configuration to the workspace level (#8891) 2024-03-05 12:01:17 -05:00
sqlez Remove if-not-else patterns (#10402) 2024-04-11 03:48:06 -04:00
sqlez_macros Move Clippy configuration to the workspace level (#8891) 2024-03-05 12:01:17 -05:00
story Enable clippy::eq_op (#9369) 2024-03-14 17:05:07 -04:00
storybook windows: Support compiling with MinGW toolchain (part 2) (#9843) 2024-03-28 10:40:07 -07:00
sum_tree Compute scrollbar markers asynchronously (#10080) 2024-04-03 12:21:17 +02:00
tab_switcher Add preview tabs (#9125) 2024-04-11 23:09:12 +02:00
task task: use full task label to distinguish a terminal (#10469) 2024-04-12 13:25:46 +02:00
tasks_ui task: Allow Rerun action to override properties of task being reran (#10468) 2024-04-12 12:44:50 +02:00
telemetry_events Extract SemanticVersion into its own crate (#9956) 2024-03-29 12:11:57 -04:00
terminal task: use full task label to distinguish a terminal (#10469) 2024-04-12 13:25:46 +02:00
terminal_view task: use full task label to distinguish a terminal (#10469) 2024-04-12 13:25:46 +02:00
text Improve the ergonomics of creating local buffers (#10347) 2024-04-10 08:32:51 -06:00
theme Add the ability for extensions to provide language settings (#10296) 2024-04-08 19:17:12 -04:00
theme_importer themes: Add pane_group.border color (#9986) 2024-04-01 12:07:26 -04:00
theme_selector Add support for applying theme after extension is installed (#9529) 2024-03-20 11:13:58 -04:00
time_format notification panel: rework time formatting (#8997) 2024-03-12 21:02:04 -06:00
ui Allow hovering over tooltips in git blame sidebar (#10466) 2024-04-12 11:47:32 +02:00
ui_text_field Update TextField (#10415) 2024-04-11 10:03:36 -04:00
util Semantic Index (#10329) 2024-04-12 11:40:59 -06:00
vcs_menu Refactor workspace notifications to use explicit NotificationId type (#10342) 2024-04-10 17:21:23 -04:00
vim Make BufferSearch less wide (#10459) 2024-04-11 23:07:29 -06:00
welcome Add preview tabs (#9125) 2024-04-11 23:09:12 +02:00
workspace preview tabs: Fix tab selection getting out of sync (#10478) 2024-04-12 17:20:30 +02:00
worktree Revert "Revert "language: Remove buffer fingerprinting (#9007)"" (#9671) 2024-04-12 18:40:35 +02:00
zed WIP: remoting (#10085) 2024-04-11 15:36:35 -06:00
zed_actions Move Clippy configuration to the workspace level (#8891) 2024-03-05 12:01:17 -05:00