Files

 multi-query-attention

/

examples

/

Copy path

Directory actions

More options

Directory actions

More options

Latest commit

loubnabnl

Revert "add humaneval generations using a server"

Nov 14, 2023

7c325cd · Nov 14, 2023

History

History

This branch is 177 commits ahead of, 4497 commits behind NVIDIA/Megatron-LM:main.

#84

/

examples

/

Top

Folders and files

Name		Name	Last commit message	Last commit date
parent directory ..
detxoify_lm		detxoify_lm	Move pipeline parallel functionality into core with associated changes.	Mar 23, 2023
msdp		msdp	fix an argument	Jan 22, 2022
sc21		sc21	more updates on examples	Aug 19, 2021
evaluate_retriever_nq.sh		evaluate_retriever_nq.sh	Checkpoint a set number of invidividual Transformer layers	Aug 19, 2021
evaluate_zeroshot_gpt.sh		evaluate_zeroshot_gpt.sh	Checkpoint a set number of invidividual Transformer layers	Aug 19, 2021
finetune_bigcode_model.slurm		finetune_bigcode_model.slurm	Finetune StarCoder Megatron	May 16, 2023
finetune_mnli_distributed.sh		finetune_mnli_distributed.sh	Checkpoint a set number of invidividual Transformer layers	Aug 19, 2021
finetune_race_distributed.sh		finetune_race_distributed.sh	Checkpoint a set number of invidividual Transformer layers	Aug 19, 2021
finetune_retriever_distributed.sh		finetune_retriever_distributed.sh	Adding readme	Jun 10, 2021
finetune_starcoderplus.slurm		finetune_starcoderplus.slurm	Create finetune_starcoderplus.slurm	Jun 12, 2023
merge_mp_bert.sh		merge_mp_bert.sh	Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP	Nov 12, 2020
pretrain_bert.sh		pretrain_bert.sh	Fix pretraining examples	Feb 17, 2023
pretrain_bert_distributed.sh		pretrain_bert_distributed.sh	Fix distributed pretraining examples	Feb 17, 2023
pretrain_bert_distributed_with_mp.sh		pretrain_bert_distributed_with_mp.sh	Fix distributed pretraining examples	Feb 17, 2023
pretrain_bigcode_1b.slurm		pretrain_bigcode_1b.slurm	Create pretrain_bigcode_1b.slurm	Jul 3, 2023
pretrain_bigcode_3b.slurm		pretrain_bigcode_3b.slurm	Create pretrain_bigcode_3b.slurm	Jul 3, 2023
pretrain_bigcode_7b.slurm		pretrain_bigcode_7b.slurm	Create pretrain_bigcode_7b.slurm	Jun 28, 2023
pretrain_bigcode_model.slurm		pretrain_bigcode_model.slurm	Update slurm script	May 12, 2023
pretrain_gpt.sh		pretrain_gpt.sh	Fix pretraining examples	Feb 17, 2023
pretrain_gpt3_175B.sh		pretrain_gpt3_175B.sh	Checkpoint a set number of invidividual Transformer layers	Aug 19, 2021
pretrain_gpt_1B_santacoder.sh		pretrain_gpt_1B_santacoder.sh	Fix train-iters typo & format script (#74)	Jul 24, 2023
pretrain_gpt_distributed.sh		pretrain_gpt_distributed.sh	Fix distributed pretraining examples	Feb 17, 2023
pretrain_gpt_distributed_with_mp.sh		pretrain_gpt_distributed_with_mp.sh	Fix distributed pretraining examples	Feb 17, 2023
pretrain_gpt_multilingual.sh		pretrain_gpt_multilingual.sh	add multi-validation for gpt training (#32)	Mar 21, 2023
pretrain_ict.sh		pretrain_ict.sh	updated script as well	Feb 15, 2021
pretrain_t5.sh		pretrain_t5.sh	Fix pretraining examples	Feb 17, 2023
pretrain_t5_distributed.sh		pretrain_t5_distributed.sh	Fix distributed pretraining examples	Feb 17, 2023
pretrain_t5_distributed_with_mp.sh		pretrain_t5_distributed_with_mp.sh	Fix distributed pretraining examples	Feb 17, 2023
run_text_generation_server_345M.sh		run_text_generation_server_345M.sh	Fix GPT text generation	Mar 9, 2023
run_text_generation_server_345M_8_tensor_parallel.sh		run_text_generation_server_345M_8_tensor_parallel.sh	Update text generation section of README. Fix line continuation in ex…	Oct 8, 2021
View all files