CodedotAl/gpt-code-clippy
1
1
Fork 0
mirror of https://github.com/CodedotAl/gpt-code-clippy.git synced 2024-09-11 15:05:44 +03:00
Code Issues Projects Releases Wiki Activity
120 Commits 16 Branches 0 Tags 18 MiB
origin/camera-ready
Commit Graph

120 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
taisazero
d70d95082d Added file to download licenses 2021-07-17 07:40:33 +00:00
ncoop57
efcf5fcde5 Merge branch 'camera-ready' of github.com:ncoop57/gpt-code-clippy into camera-ready 2021-07-15 15:59:45 -04:00
ncoop57
a2e6c08a3c WIP on human eval evaluation 2021-07-15 15:59:42 -04:00
Mrinal Mathur
a3a90effed
Merge pull request #49 from ncoop57/Mrinal18-camera-ready-1 
Adding documentation on usage
 2021-07-16 01:25:46 +05:30
ncoop57
f49fda52e2 Merge branch 'main' of github.com:ncoop57/gpt-code-clippy into main 2021-07-15 19:04:57 +00:00
ncoop57
4ffcfd5528 Add new downloading scripts 2021-07-15 19:04:54 +00:00
Arun Raja
d3b5b8f530
Merge pull request #52 from ncoop57/13b 
added partitions.py for 1.3b parallel training
 2021-07-15 22:59:22 +08:00
arunraja-hub
2521f896f4 added partitions.py for 1.3b parallel training 2021-07-15 14:58:29 +00:00
Arun Raja
9ba71b77d5
Merge pull request #51 from ncoop57/13b 
shell script for 1.3b streaming [WIP]
 2021-07-15 21:44:04 +08:00
arunraja-hub
65f6279f47 shell script for 1.3b streaming 2021-07-15 13:42:11 +00:00
Mrinal Mathur
0d63ea7fa9
Updated training file information 2021-07-15 17:46:38 +05:30
arampacha
d88e51f8a1 adds apps dataset loading and reindent scripts 2021-07-15 14:13:24 +03:00
arampacha
1cbe0edf1e updates run_clm_streaming_flax.py 2021-07-15 14:13:24 +03:00
Mrinal Mathur
c5966efd86
Adding documentation on usage 2021-07-15 15:12:58 +05:30
ncoop57
9c5c7aa4e1 Even more reorganizing 2021-07-14 20:36:03 -04:00
ncoop57
8855f0c40e More reorganization 2021-07-14 20:24:37 -04:00
ncoop57
51dc4c3852 Update readme with outline 2021-07-14 20:22:13 -04:00
ncoop57
6ccb72c0ad Merge branch 'docs' of github.com:ncoop57/gpt-code-clippy into camera-ready 2021-07-14 20:12:47 -04:00
ncoop57
745372a744 SOme initial clean up 2021-07-14 20:12:26 -04:00
arampacha
30f2ee4127
reinstantiate opt_state recursively when there are nested lists (#47) 2021-07-15 03:09:40 +03:00
ncoop57
6d4c9bfa23 Convert variable names to hashes for smaller memory footprint 2021-07-14 23:58:09 +00:00
Ben Trevett
cb64f77b52
Merge pull request #46 from ncoop57/deduplication 
add deduplication for streaming and fixed readme
 2021-07-14 15:36:18 +01:00
bentrevett
8c7fb0ed99 added streaming duplicate remover (probably v. slow) 2021-07-14 15:34:35 +01:00
bentrevett
07aacc88da update readme 2021-07-14 15:13:04 +01:00
bentrevett
5502b19552 update readme 2021-07-14 15:12:16 +01:00
Ben Trevett
a4abfc85b9
Merge pull request #45 from ncoop57/deduplication 
change deduplication to use huggingface dataset unique method
 2021-07-14 15:09:27 +01:00
bentrevett
0eb1dd5960 remove streaming arg 2021-07-14 15:08:42 +01:00
bentrevett
706381e23d change deduplication to use HuggingFace datasets 2021-07-14 15:08:30 +01:00
Ben Trevett
4b2b7ca00c
Merge pull request #44 from ncoop57/fix_dedup_parallel 
fix deduplication_parallel
 2021-07-14 13:26:12 +01:00
bentrevett
7579ec9697 fix deduplication_parallel 2021-07-14 13:25:40 +01:00
ncoop57
b1354dd181 Add parallel, but buggy version of dedup script 2021-07-14 12:01:38 +00:00
arampacha
3aef752fce
upd clm script (#43) 
* update resuming behaviour

* adds temporary hack to enable resuming w multisteps and adafactor
 2021-07-14 00:52:59 +03:00
bentrevett
47bb3895bc add type hints 2021-07-13 22:47:54 +01:00
Ben Trevett
309c4095c0
Merge pull request #42 from ncoop57/deduplication 
use threaded=True for the lm_dataformat.Reader
 2021-07-13 22:27:34 +01:00
bentrevett
0ba2261d29 threaded the lm_dataformat.Reader 2021-07-13 22:26:49 +01:00
Ben Trevett
dac5b42c2d
Merge pull request #41 from ncoop57/deduplication 
update deduplication code to write deduplicated data to folder
 2021-07-13 17:56:50 +01:00
bentrevett
08bb88ad4b update readme 2021-07-13 17:55:50 +01:00
bentrevett
f0160ebbbe improve deduplication 
- remove code_hash from DocumentID (not needed)
- can now write non-duplicate files to given `output_dir`
- number of examples per file given by `archive_commit_frequency`
 2021-07-13 17:53:01 +01:00
arampacha
92539be4cb
Training script update (#40) 
* adds cleaner streaming script

* resume from checkpoint w/ MultiSteps

* adds gradient clipping

* upd run_clm_flax.py
 2021-07-13 18:38:04 +03:00
Santiago
cdbf1572d6 WIP: EDA 
feat: add data preprocessing
TODO: implement keyword detection and deduplication stats
 2021-07-13 13:26:10 +00:00
arunraja-hub
4aabbacbe9 bs=1 and corrected save checkpoint function 2021-07-12 17:28:31 +00:00
ncoop57
73aa110de9 Add model card and data sheet templates 2021-07-12 13:15:22 -04:00
ncoop57
41a7423ca4 Add runner for 1.3b model 2021-07-12 00:00:04 +00:00
ncoop57
b110dd905b Increase saving lim 2021-07-11 23:58:12 +00:00
ncoop57
127caba3d8 Update requirments 2021-07-11 23:45:58 +00:00
ncoop57
1e29442203 Rename to make clearer 2021-07-11 23:39:20 +00:00
ncoop57
489e424169 Change weight decay to match gpt codex 2021-07-11 23:00:40 +00:00
ncoop57
acead42c66 Update model name 2021-07-11 22:51:57 +00:00
ncoop57
5e80c79576 Update hyperparams 2021-07-11 22:50:59 +00:00
ncoop57
a3b9f56a02 Add requirements file and run config script 2021-07-11 22:38:22 +00:00