Commit Graph

58 Commits

Author SHA1 Message Date
KCaverly
9781047156 move truncation to parsing step leveraging the EmbeddingProvider trait 2023-08-30 12:13:26 -04:00
KCaverly
76caea80f7 add should_truncate to embedding providers 2023-08-30 11:58:45 -04:00
KCaverly
e377ada1a9 added token count to documents during parsing 2023-08-30 11:05:46 -04:00
KCaverly
a7e6a65deb reindex files in the background after they have not been edited for 10 minutes
Co-authored-by: Max <max@zed.dev>
2023-08-29 17:14:44 -04:00
KCaverly
4f8b95cf0d add proper handling for open ai rate limit delays 2023-08-29 15:44:51 -04:00
Max Brunsfeld
404f76739c Format let-else statements 2023-08-25 10:11:32 -07:00
KCaverly
ee97bc54cf cleaned up warnings 2023-08-25 10:38:01 +02:00
KCaverly
131950f670 add handling for Added file events to semantic index 2023-08-24 18:40:08 +02:00
KCaverly
a892a51ec3 update initialize project call to accomodate for test scenarios
Co-authored-by: Piotr <piotr@zed.dev>
2023-08-24 13:46:43 +02:00
KCaverly
a1519e4c38 move semantic search project intialization to a subscribe event for workspace created
Co-authored-by: Piotr <piotr@zed.dev>
2023-08-24 13:14:19 +02:00
KCaverly
e8e7b294d8 add delete files operation for remaining files in database not included in current worktree
Co-authored-by: Piotr <piotr@zed.dev>
2023-08-24 12:49:20 +02:00
KCaverly
afe0e74868 remove worktree_file_mtimes in state as it is no longer used
Co-authored-by: Piotr <piotr@zed.dev>
2023-08-24 12:42:41 +02:00
KCaverly
aa07872a24 accomodate for duplicate entries in indexing queue
Co-authored-by: Piotr <piotr@zed.dev>
2023-08-24 12:36:43 +02:00
KCaverly
3f9f742530 update rate limiting embeddings strategy to delay less 2023-08-24 11:45:52 +02:00
KCaverly
e42b9e910e fix async calls on project updated entries to ensure that all files are updating appropriately 2023-08-23 22:28:30 +02:00
KCaverly
09fd99b1e3 moved semantic_index project intialization to queue and channel method 2023-08-23 15:09:15 +02:00
KCaverly
328b7e523c reorganized to stop the race 2023-08-22 15:01:21 +02:00
KCaverly
aabdfa210f working on initialization + index breakup 2023-08-22 14:45:27 +02:00
KCaverly
ced2b2aec3 reworked ProjectState to include additional context 2023-08-22 11:58:48 +02:00
KCaverly
3d89cd10a4 added sha1 encoding for each document 2023-08-21 16:35:57 +02:00
Piotr Osiewicz
c68b518aec chore: fix compiler warning
Co-authored-by: Kyle <kyle@zed.dev>
2023-08-21 13:31:45 +02:00
Piotr Osiewicz
61041b0cd1 Do not attempt to reindex a file if previous attempts have failed.
Add doc comment to JobHandle

Co-authored-by: Kyle <kyle@zed.dev>
2023-08-21 13:23:11 +02:00
Piotr Osiewicz
1a88444f2f Increment job counter on JobClient::new
Co-authored-by: Kyle <kyle@zed.dev>
2023-08-21 13:00:56 +02:00
KCaverly
def215af9f update job handle to ensure file count is consistent
Co-authored-by: Piotr <piotr@zed.dev>
2023-08-21 12:47:43 +02:00
KCaverly
1cae4758cc manage for edge case in which file documents are larger than the allowable limit 2023-08-21 11:29:45 +02:00
Piotr Osiewicz
e2b8e6ce63 chore: fix compiler warnings 2023-08-09 01:16:24 +02:00
Piotr Osiewicz
31fb503418 Rewrite permission queries (it no longer blocks)
Co-authored-by: Kyle <kyle@zed.dev>
Co-authored-by: Max <max@zed.dev>
2023-08-03 20:26:56 +02:00
KCaverly
8dd330a160 add semantic search prompt for the indexing permission on first search
Co-authored-by: Piotr <piotr@zed.dev>
2023-08-03 12:52:20 -04:00
KCaverly
300c693d55 catchup with main 2023-08-01 10:40:38 -04:00
KCaverly
e221f23018 add support for markdown files to semantic search 2023-08-01 10:30:34 -04:00
KCaverly
9a50b43eaa add templating languages html, erb, heex, svelte as entire parseable file types 2023-07-31 21:03:02 -04:00
KCaverly
599f674827 add php support for semantic search 2023-07-31 16:36:09 -04:00
KCaverly
89edb3d1b5 fix templating bug for parseable entire files 2023-07-31 11:41:18 -04:00
KCaverly
ca4e21881e add ruby support for semantic search 2023-07-31 10:54:30 -04:00
KCaverly
a5dd8dd0a9 add lua embedding query for semantic search 2023-07-31 10:02:28 -04:00
Kirill Bulatov
cf6e524c9a Make project search includes and excludes more user-friendly
Allow search results that start with the include/exclude path part
2023-07-28 12:56:44 +03:00
KCaverly
fbede4a5a3 removed old code 2023-07-26 17:11:30 -04:00
KCaverly
5c48729c7c managed for small batch size edge case in semantic search 2023-07-26 16:56:41 -04:00
KCaverly
98fde36834 batch search queries in the vector database 2023-07-26 16:36:39 -04:00
KCaverly
0b61c93a25 ensure semantic search is not enabled on stable 2023-07-26 10:22:33 -04:00
KCaverly
394a105639 fix warnings 2023-07-26 10:03:30 -04:00
KCaverly
ca6f7d8a80 add worktree previously indexed functionality to vector db 2023-07-26 09:17:04 -04:00
KCaverly
75999204ad update project search to only show semantic button visible with semantic_index enabled 2023-07-25 16:26:37 -04:00
KCaverly
cdceddd2cc update semantic index tests for elixir 2023-07-25 15:20:35 -04:00
KCaverly
97c3d97792 update semantic index tests for cpp 2023-07-25 13:30:38 -04:00
KCaverly
c86096a886 update semantic index tests for javascript 2023-07-25 10:38:37 -04:00
Max Brunsfeld
81b05f2a08 Optimize glob filtering of semantic search
Co-authored-by: Kyle <kyle@zed.dev>
2023-07-20 14:23:11 -07:00
KCaverly
e02d6bc0d4 add glob filtering functionality to semantic search 2023-07-20 13:46:27 -04:00
KCaverly
efe973ebe2 add embedding query for json with nested arrays and strings
Co-authored-by: maxbrunsfeld <max@zed.dev>
2023-07-19 16:52:44 -04:00
KCaverly
9809ec3d70 update treesitter parsing to accomodate for collapsed nested functions
Co-authored-by: maxbrunsfeld <max@zed.dev>
2023-07-19 15:47:05 -04:00