首页 星云 工具 资源 星选 资讯 热门工具
:

PDF转图片 完全免费 小红书视频下载 无水印 抖音视频下载 无水印 数字星空

gpt-neox-main.zip

人工智能 51.27MB 18 需要积分: 1
立即下载

资源介绍:

gpt-neox-main.zip
[![GitHub issues](https://img.shields.io/github/issues/EleutherAI/gpt-neox)](https://github.com/EleutherAI/gpt-neox/issues) [Weights & Biases monitoring](https://wandb.ai/eleutherai/neox) # GPT-NeoX This repository records [EleutherAI](https://www.eleuther.ai)'s library for training large-scale language models on GPUs. Our current framework is based on NVIDIA's [Megatron Language Model](https://github.com/NVIDIA/Megatron-LM) and has been augmented with techniques from [DeepSpeed](https://www.deepspeed.ai) as well as some novel optimizations. We aim to make this repo a centralized and accessible place to gather techniques for training large-scale autoregressive language models, and accelerate research into large-scale training. This library is in widespread use in [academic, industry, and government labs](https://github.com/EleutherAI/gpt-neox#adoption-and-publications), including by researchers at Oak Ridge National Lab, CarperAI, Stability AI, Together.ai, Korea University, Carnegie Mellon University, and the University of Tokyo among others. Uniquely among similar libraries GPT-NeoX supports a wide variety of systems and hardwares, including launching via Slurm, MPI, and the IBM Job Step Manager, and has been run at scale on [AWS](https://aws.amazon.com/), [CoreWeave](https://www.coreweave.com/), [ORNL Summit](https://www.olcf.ornl.gov/summit/), [ORNL Frontier](https://www.olcf.ornl.gov/frontier/), [LUMI](https://www.lumi-supercomputer.eu/), and others. **If you are not looking to train models with billions of parameters from scratch, this is likely the wrong library to use. For generic inference needs, we recommend you use the Hugging Face `transformers` library instead which supports GPT-NeoX models.** ## Why GPT-NeoX? GPT-NeoX leverages many of the same features and technologies as the popular Megatron-DeepSpeed library but with substantially increased usability and novel optimizations. Major features include: * Distributed training with ZeRO and 3D parallelism * A wide variety of systems and hardwares, including launching via Slurm, MPI, and the IBM Job Step Manager, and has been run at scale on [AWS](https://aws.amazon.com/), [CoreWeave](https://www.coreweave.com/), Oak Ridge's [Summit](https://www.olcf.ornl.gov/summit/) and [Frontier](https://www.olcf.ornl.gov/frontier/), [Pacific Northwest National Laboratory](https://hpc.pnl.gov/index.shtml), Argonne's [Polaris](https://docs.alcf.anl.gov/polaris/data-science-workflows/applications/gpt-neox/), [LUMI](https://www.lumi-supercomputer.eu/), and more. * Cutting edge architectural innovations including rotary and alibi positional embeddings, parallel feedforward attention layers, and flash attention. * Predefined configurations for popular architectures including Pythia, PaLM, Falcon, and LLaMA 1 \& 2 * Curriculum Learning * Easy connections with the open source ecosystem, including Hugging Face's [tokenizers](https://github.com/huggingface/tokenizers) and [transformers](https://github.com/huggingface/transformers/) libraries, logging via [WandB](https://wandb.ai/site), and evaluation via our [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness). ## News **[8/10/2023]** We now support checkpointing with AWS S3! Activate with the `s3_path` config option (for more detail, see [the PR](https://github.com/EleutherAI/gpt-neox/pull/1010)) **[9/20/2023]** As of https://github.com/EleutherAI/gpt-neox/pull/1035, we have deprecated Flash Attention 0.x and 1.x, and migrated support to Flash Attention 2.x. We don't believe this will cause problems, but if you have a specific use-case that requires old flash support using the latest GPT-NeoX, please raise an issue. **[8/10/2023]** We have experimental support for LLaMA 2 and Flash Attention v2 supported in our [math-lm](https://github.com/EleutherAI/math-lm) project that will be upstreamed later this month. **[5/17/2023]** After fixing some miscellaneous bugs we now fully support bf16. **[4/11/2023]** We have upgraded our Flash Attention implementation to now support Alibi positional embeddings. **[3/9/2023]** We have released GPT-NeoX 2.0.0, an upgraded version built on the latest DeepSpeed which will be regularly synced with going forward. ## Versions Prior to 3/9/2023, GPT-NeoX relied on [DeeperSpeed](https://github.com/EleutherAI/DeeperSpeed), which was based on an old version of DeepSpeed (0.3.15). In order to migrate to the latest upstream DeepSpeed version while allowing users to access the old versions of GPT-NeoX and DeeperSpeed, we have introduced two versioned releases for both libraries: - Version 2.0 of [GPT-NeoX](https://github.com/EleutherAI/gpt-neox/releases/tag/v2.0) and [DeeperSpeed](https://github.com/EleutherAI/DeeperSpeed/releases/tag/v2.0) are the latest versions built on the latest DeepSpeed, and will be maintained going forward. - Version 1.0 of [GPT-NeoX](https://github.com/EleutherAI/gpt-neox/releases/tag/v1.0) and [DeeperSpeed](https://github.com/EleutherAI/DeeperSpeed/releases/tag/v1.0) maintain snapshots of the old stable versions that [GPT-NeoX-20B](https://arxiv.org/abs/2204.06745) and the [Pythia Suite](https://github.com/EleutherAI/pythia) were trained on. # Contents - [GPT-NeoX](#gpt-neox) * [Why GPT-NeoX?](#why-gpt-neox) * [News](#news) * [Versions](#versions) - [Contents](#contents) - [Quick Start](#quick-start) * [Environment and Dependencies](#environment-and-dependencies) + [Host Setup](#host-setup) + [Flash Attention](#flash-attention) + [Multi-Node Launching](#multi-node-launching) + [Containerized Setup](#containerized-setup) * [Usage](#usage) - [Configuration](#configuration) * [Mixture of Experts](#mixture-of-experts) - [Datasets](#datasets) * [Preconfigured Datasets](#preconfigured-datasets) * [Using Custom Data](#using-custom-data) - [Training and Finetuning](#training-and-finetuning) * [Pretrained Models](#pretrained-models) + [GPT-NeoX-20B](#gpt-neox-20b) + [Pythia](#pythia) + [Polyglot](#polyglot) - [Inference](#inference) - [Evaluation](#evaluation) - [Exporting to Hugging Face](#exporting-to-hugging-face) - [Monitoring](#monitoring) * [Weights and Biases](#weights-and-biases) * [TensorBoard](#tensorboard) - [Running on multi-node](#running-on-multi-node) - [Profiling](#profiling) - [Adoption and Publications](#adoption-and-publications) * [Publications](#publications) * [Models](#models) + [English LLMs](#english-llms) + [Non-English LLMs](#non-english-llms) + [Code Models](#code-models) + [Other Modalities](#other-modalities) - [Administrative Notes](#administrative-notes) * [Citing GPT-NeoX](#citing-gpt-neox) * [Contributing](#contributing) * [Licensing](#licensing) * [Acknowledgements](#acknowledgements) # Quick Start ## Environment and Dependencies ### Host Setup First make sure you are in an environment with Python 3.8 with an appropriate version of PyTorch 1.8 or later installed. **Note:** Some of the libraries that GPT-NeoX depends on have not been updated to be compatible with Python 3.10+. Python 3.9 appears to work, but this codebase has been developed and tested for Python 3.8. To install the remaining basic dependencies, run: ```bash pip install -r requirements/requirements.txt pip install -r requirements/requirements-wandb.txt # optional, if logging using WandB pip install -r requirements/requirements-tensorboard.txt # optional, if logging via tensorboard python ./megatron/fused_kernels/setup.py install # optional, if using fused kernels ``` from the repository root. > [!Warning] > Our codebase relies on [DeeperSpeed](https://github.com/EleutherAI/DeeperSpeed), our fork of the [DeepSpeed](https://github.com/microsoft/DeepSpeed) library with some added changes. We strongly recommend using Anaconda, a virtual machine, or some other form of environment isol

资源文件列表:

gpt-neox-main.zip 大约有292个文件
  1. gpt-neox-main/
  2. gpt-neox-main/.idea/
  3. gpt-neox-main/.idea/.gitignore 50B
  4. gpt-neox-main/.idea/gpt-neox-main.iml 567B
  5. gpt-neox-main/.idea/inspectionProfiles/
  6. gpt-neox-main/.idea/inspectionProfiles/profiles_settings.xml 174B
  7. gpt-neox-main/.idea/misc.xml 292B
  8. gpt-neox-main/.idea/modules.xml 285B
  9. gpt-neox-main/.idea/workspace.xml 2.06KB
  10. gpt-neox-main/gpt-neox-main/
  11. gpt-neox-main/gpt-neox-main/.clang-format 4.4KB
  12. gpt-neox-main/gpt-neox-main/.dockerignore 17B
  13. gpt-neox-main/gpt-neox-main/.github/
  14. gpt-neox-main/gpt-neox-main/.github/CODEOWNERS 19B
  15. gpt-neox-main/gpt-neox-main/.github/ISSUE_TEMPLATE/
  16. gpt-neox-main/gpt-neox-main/.github/ISSUE_TEMPLATE/bug_report.md 712B
  17. gpt-neox-main/gpt-neox-main/.github/ISSUE_TEMPLATE/feature_request.md 608B
  18. gpt-neox-main/gpt-neox-main/.github/workflows/
  19. gpt-neox-main/gpt-neox-main/.github/workflows/coverity_scan.yml 1.96KB
  20. gpt-neox-main/gpt-neox-main/.github/workflows/cpu_ci.yml 1017B
  21. gpt-neox-main/gpt-neox-main/.github/workflows/cpu_ci_dispatch.yml 438B
  22. gpt-neox-main/gpt-neox-main/.github/workflows/cpu_ci_on_pr.yml 425B
  23. gpt-neox-main/gpt-neox-main/.github/workflows/docker_build.yml 1.16KB
  24. gpt-neox-main/gpt-neox-main/.github/workflows/pull_request.yml 1.36KB
  25. gpt-neox-main/gpt-neox-main/.gitignore 2.05KB
  26. gpt-neox-main/gpt-neox-main/.pre-commit-config.yaml 1.35KB
  27. gpt-neox-main/gpt-neox-main/CITATION.cff 2.02KB
  28. gpt-neox-main/gpt-neox-main/ckpts/
  29. gpt-neox-main/gpt-neox-main/ckpts/20B_tokenizer.json 2.11MB
  30. gpt-neox-main/gpt-neox-main/configs/
  31. gpt-neox-main/gpt-neox-main/configs/1-3B.yml 2.32KB
  32. gpt-neox-main/gpt-neox-main/configs/125M-dmoe.yml 2.47KB
  33. gpt-neox-main/gpt-neox-main/configs/125M-json.yml 1.69KB
  34. gpt-neox-main/gpt-neox-main/configs/125M-moe.yml 2.47KB
  35. gpt-neox-main/gpt-neox-main/configs/125M.yml 2.35KB
  36. gpt-neox-main/gpt-neox-main/configs/125M_my.yml 2.35KB
  37. gpt-neox-main/gpt-neox-main/configs/13B.yml 2.32KB
  38. gpt-neox-main/gpt-neox-main/configs/175B.yml 2.32KB
  39. gpt-neox-main/gpt-neox-main/configs/19M.yml 2.14KB
  40. gpt-neox-main/gpt-neox-main/configs/2-7B.yml 2.32KB
  41. gpt-neox-main/gpt-neox-main/configs/20B.yml 3KB
  42. gpt-neox-main/gpt-neox-main/configs/350M.yml 2.32KB
  43. gpt-neox-main/gpt-neox-main/configs/49M.yml 2.15KB
  44. gpt-neox-main/gpt-neox-main/configs/6-7B.yml 2.32KB
  45. gpt-neox-main/gpt-neox-main/configs/760M.yml 2.32KB
  46. gpt-neox-main/gpt-neox-main/configs/800M.yml 1.93KB
  47. gpt-neox-main/gpt-neox-main/configs/autotuning_configs/
  48. gpt-neox-main/gpt-neox-main/configs/autotuning_configs/small_tune.json 1.86KB
  49. gpt-neox-main/gpt-neox-main/configs/autotuning_configs/tune.json 1.83KB
  50. gpt-neox-main/gpt-neox-main/configs/autotuning_configs/tune_1-3B.json 2.01KB
  51. gpt-neox-main/gpt-neox-main/configs/autotuning_configs/tune_6-7B.json 1.69KB
  52. gpt-neox-main/gpt-neox-main/configs/bf16_125M.yml 2.11KB
  53. gpt-neox-main/gpt-neox-main/configs/bnb_125M.yml 2.17KB
  54. gpt-neox-main/gpt-neox-main/configs/cpu_mock_config.yml 186B
  55. gpt-neox-main/gpt-neox-main/configs/docker/
  56. gpt-neox-main/gpt-neox-main/configs/docker/pythia-paths.yml 496B
  57. gpt-neox-main/gpt-neox-main/configs/eleutherai_cluster.yml 1.1KB
  58. gpt-neox-main/gpt-neox-main/configs/finetuning_configs/
  59. gpt-neox-main/gpt-neox-main/configs/finetuning_configs/6-9B.yml 1.96KB
  60. gpt-neox-main/gpt-neox-main/configs/gen_docs.py 3.14KB
  61. gpt-neox-main/gpt-neox-main/configs/gmlp_small.yml 1.74KB
  62. gpt-neox-main/gpt-neox-main/configs/llama/
  63. gpt-neox-main/gpt-neox-main/configs/llama/13B.yml 628B
  64. gpt-neox-main/gpt-neox-main/configs/llama/30B.yml 628B
  65. gpt-neox-main/gpt-neox-main/configs/llama/65B.yml 628B
  66. gpt-neox-main/gpt-neox-main/configs/llama/7B.yml 628B
  67. gpt-neox-main/gpt-neox-main/configs/llama/README.md 678B
  68. gpt-neox-main/gpt-neox-main/configs/llama/train_config.yml 1.58KB
  69. gpt-neox-main/gpt-neox-main/configs/llama2/
  70. gpt-neox-main/gpt-neox-main/configs/llama2/13B.yml 628B
  71. gpt-neox-main/gpt-neox-main/configs/llama2/70B.yml 751B
  72. gpt-neox-main/gpt-neox-main/configs/llama2/7B.yml 628B
  73. gpt-neox-main/gpt-neox-main/configs/llama2/codellama_34B.yml 829B
  74. gpt-neox-main/gpt-neox-main/configs/llama2/codellama_7B.yml 808B
  75. gpt-neox-main/gpt-neox-main/configs/llemma/
  76. gpt-neox-main/gpt-neox-main/configs/llemma/34B.yml 2.61KB
  77. gpt-neox-main/gpt-neox-main/configs/llemma/7B.yml 2.51KB
  78. gpt-neox-main/gpt-neox-main/configs/local_setup.yml 1.2KB
  79. gpt-neox-main/gpt-neox-main/configs/mamba/
  80. gpt-neox-main/gpt-neox-main/configs/mamba/mamba-1.4B.yml 628B
  81. gpt-neox-main/gpt-neox-main/configs/mamba/mamba-130M.yml 627B
  82. gpt-neox-main/gpt-neox-main/configs/mamba/mamba-2.8B.yml 628B
  83. gpt-neox-main/gpt-neox-main/configs/mamba/mamba-370M.yml 628B
  84. gpt-neox-main/gpt-neox-main/configs/mamba/mamba-790M.yml 628B
  85. gpt-neox-main/gpt-neox-main/configs/mistral/
  86. gpt-neox-main/gpt-neox-main/configs/mistral/7B.yml 1.32KB
  87. gpt-neox-main/gpt-neox-main/configs/neox_arguments.md 42.76KB
  88. gpt-neox-main/gpt-neox-main/configs/pythia/
  89. gpt-neox-main/gpt-neox-main/configs/pythia/1-4B.yml 1.79KB
  90. gpt-neox-main/gpt-neox-main/configs/pythia/12B.yml 1.84KB
  91. gpt-neox-main/gpt-neox-main/configs/pythia/14M.yml 2.26KB
  92. gpt-neox-main/gpt-neox-main/configs/pythia/160M.yml 1.79KB
  93. gpt-neox-main/gpt-neox-main/configs/pythia/1B.yml 1.84KB
  94. gpt-neox-main/gpt-neox-main/configs/pythia/2-8B.yml 1.85KB
  95. gpt-neox-main/gpt-neox-main/configs/pythia/31M.yml 2.25KB
  96. gpt-neox-main/gpt-neox-main/configs/pythia/410M.yml 1.79KB
  97. gpt-neox-main/gpt-neox-main/configs/pythia/6-9B.yml 1.82KB
  98. gpt-neox-main/gpt-neox-main/configs/pythia/70M.yml 1.79KB
  99. gpt-neox-main/gpt-neox-main/configs/README.md 12.15KB
  100. gpt-neox-main/gpt-neox-main/configs/rwkv/
  101. gpt-neox-main/gpt-neox-main/configs/rwkv/170M.yml 2.36KB
  102. gpt-neox-main/gpt-neox-main/configs/slurm_125M.yml 1.63KB
  103. gpt-neox-main/gpt-neox-main/configs/slurm_local.json 305B
  104. gpt-neox-main/gpt-neox-main/configs/slurm_local.yml 356B
  105. gpt-neox-main/gpt-neox-main/configs/sparse.yml 542B
  106. gpt-neox-main/gpt-neox-main/configs/text_generation.yml 494B
  107. gpt-neox-main/gpt-neox-main/CONTRIBUTING.md 4.62KB
  108. gpt-neox-main/gpt-neox-main/data/
  109. gpt-neox-main/gpt-neox-main/data/openwebtext2_sample.jsonl 125.51MB
  110. gpt-neox-main/gpt-neox-main/deepy.py 1.31KB
  111. gpt-neox-main/gpt-neox-main/docker-compose-dockerhub.yml 545B
  112. gpt-neox-main/gpt-neox-main/docker-compose.yml 589B
  113. gpt-neox-main/gpt-neox-main/Dockerfile 3.76KB
  114. gpt-neox-main/gpt-neox-main/eval.py 2.6KB
  115. gpt-neox-main/gpt-neox-main/eval_tasks/
  116. gpt-neox-main/gpt-neox-main/eval_tasks/eval_adapter.py 19.82KB
  117. gpt-neox-main/gpt-neox-main/eval_tasks/__init__.py 643B
  118. gpt-neox-main/gpt-neox-main/generate.py 3.24KB
  119. gpt-neox-main/gpt-neox-main/images/
  120. gpt-neox-main/gpt-neox-main/images/memory_profiling.png 1.04MB
  121. gpt-neox-main/gpt-neox-main/images/nsight_profiling.png 472.09KB
  122. gpt-neox-main/gpt-neox-main/LICENSE 25.18KB
  123. gpt-neox-main/gpt-neox-main/MANIFEST.in 65B
  124. gpt-neox-main/gpt-neox-main/megatron/
  125. gpt-neox-main/gpt-neox-main/megatron/checkpointing.py 17.14KB
  126. gpt-neox-main/gpt-neox-main/megatron/data/
  127. gpt-neox-main/gpt-neox-main/megatron/data/blendable_dataset.py 2.56KB
  128. gpt-neox-main/gpt-neox-main/megatron/data/data_utils.py 17.63KB
  129. gpt-neox-main/gpt-neox-main/megatron/data/gpt2_dataset.py 12.54KB
  130. gpt-neox-main/gpt-neox-main/megatron/data/helpers.cpp 33.18KB
  131. gpt-neox-main/gpt-neox-main/megatron/data/indexed_dataset.py 18.79KB
  132. gpt-neox-main/gpt-neox-main/megatron/data/Makefile 279B
  133. gpt-neox-main/gpt-neox-main/megatron/data/samplers.py 6.07KB
  134. gpt-neox-main/gpt-neox-main/megatron/data/test.py 20B
  135. gpt-neox-main/gpt-neox-main/megatron/data/__init__.py 16B
  136. gpt-neox-main/gpt-neox-main/megatron/devutil.py 1.25KB
  137. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/
  138. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/compat.h 893B
  139. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/fused_rotary_positional_embedding.cpp 6.37KB
  140. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/fused_rotary_positional_embedding.h 18.63KB
  141. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/fused_rotary_positional_embedding_cuda.cu 15.36KB
  142. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/scaled_masked_softmax.cpp 3.13KB
  143. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/scaled_masked_softmax.h 23.44KB
  144. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/scaled_masked_softmax_cuda.cu 4.55KB
  145. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/scaled_upper_triang_masked_softmax.cpp 2.64KB
  146. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/scaled_upper_triang_masked_softmax.h 26.3KB
  147. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/scaled_upper_triang_masked_softmax_cuda.cu 3.37KB
  148. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/setup.py 2.92KB
  149. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/type_shim.h 21.61KB
  150. gpt-neox-main/gpt-neox-main/megatron/fused_kernels/__init__.py 5.86KB
  151. gpt-neox-main/gpt-neox-main/megatron/gradient_noise_scale/
  152. gpt-neox-main/gpt-neox-main/megatron/gradient_noise_scale/gradient_noise_scale.py 7.96KB
  153. gpt-neox-main/gpt-neox-main/megatron/gradient_noise_scale/__init__.py 53B
  154. gpt-neox-main/gpt-neox-main/megatron/initialize.py 8.38KB
  155. gpt-neox-main/gpt-neox-main/megatron/learning_rates.py 5.1KB
  156. gpt-neox-main/gpt-neox-main/megatron/logging.py 13.65KB
  157. gpt-neox-main/gpt-neox-main/megatron/model/
  158. gpt-neox-main/gpt-neox-main/megatron/model/activations.py 4.28KB
  159. gpt-neox-main/gpt-neox-main/megatron/model/fused_bias_dropout.py 1.83KB
  160. gpt-neox-main/gpt-neox-main/megatron/model/fused_layer_norm.py 4.77KB
  161. gpt-neox-main/gpt-neox-main/megatron/model/fused_rope.py 4.84KB
  162. gpt-neox-main/gpt-neox-main/megatron/model/fused_softmax.py 6.83KB
  163. gpt-neox-main/gpt-neox-main/megatron/model/gmlp.py 4.97KB
  164. gpt-neox-main/gpt-neox-main/megatron/model/gpt2_model.py 16.06KB
  165. gpt-neox-main/gpt-neox-main/megatron/model/init_functions.py 7.49KB
  166. gpt-neox-main/gpt-neox-main/megatron/model/mamba/
  167. gpt-neox-main/gpt-neox-main/megatron/model/mamba/mamba.py 14.32KB
  168. gpt-neox-main/gpt-neox-main/megatron/model/mamba/__init__.py 91B
  169. gpt-neox-main/gpt-neox-main/megatron/model/megablocks_utils.py 896B
  170. gpt-neox-main/gpt-neox-main/megatron/model/norms.py 2.89KB
  171. gpt-neox-main/gpt-neox-main/megatron/model/positional_embeddings.py 9.93KB
  172. gpt-neox-main/gpt-neox-main/megatron/model/rwkv/
  173. gpt-neox-main/gpt-neox-main/megatron/model/rwkv/v6/
  174. gpt-neox-main/gpt-neox-main/megatron/model/rwkv/v6/cuda/
  175. gpt-neox-main/gpt-neox-main/megatron/model/rwkv/v6/cuda/wkv6_cuda.cu 7.87KB
  176. gpt-neox-main/gpt-neox-main/megatron/model/rwkv/v6/cuda/wkv6_op.cpp 2.5KB
  177. gpt-neox-main/gpt-neox-main/megatron/model/rwkv/v6/rwkv.py 12.46KB
  178. gpt-neox-main/gpt-neox-main/megatron/model/rwkv/v6/__init__.py 59B
  179. gpt-neox-main/gpt-neox-main/megatron/model/rwkv/__init__.py
  180. gpt-neox-main/gpt-neox-main/megatron/model/transformer.py 49.76KB
  181. gpt-neox-main/gpt-neox-main/megatron/model/utils.py 14.12KB
  182. gpt-neox-main/gpt-neox-main/megatron/model/word_embeddings.py 9.4KB
  183. gpt-neox-main/gpt-neox-main/megatron/model/__init__.py 894B
  184. gpt-neox-main/gpt-neox-main/megatron/mpu/
  185. gpt-neox-main/gpt-neox-main/megatron/mpu/cross_entropy.py 4.69KB
  186. gpt-neox-main/gpt-neox-main/megatron/mpu/data.py 3.79KB
  187. gpt-neox-main/gpt-neox-main/megatron/mpu/initialize.py 10.87KB
  188. gpt-neox-main/gpt-neox-main/megatron/mpu/layers.py 27.37KB
  189. gpt-neox-main/gpt-neox-main/megatron/mpu/mappings.py 4.83KB
  190. gpt-neox-main/gpt-neox-main/megatron/mpu/random.py 1.53KB
  191. gpt-neox-main/gpt-neox-main/megatron/mpu/utils.py 2.71KB
  192. gpt-neox-main/gpt-neox-main/megatron/mpu/__init__.py 2.31KB
  193. gpt-neox-main/gpt-neox-main/megatron/mup_substitute.py 7.62KB
  194. gpt-neox-main/gpt-neox-main/megatron/neox_arguments/
  195. gpt-neox-main/gpt-neox-main/megatron/neox_arguments/arguments.py 54.58KB
  196. gpt-neox-main/gpt-neox-main/megatron/neox_arguments/deepspeed_args.py 11.86KB
  197. gpt-neox-main/gpt-neox-main/megatron/neox_arguments/neox_args.py 34.96KB
  198. gpt-neox-main/gpt-neox-main/megatron/neox_arguments/template.py 1.63KB
  199. gpt-neox-main/gpt-neox-main/megatron/neox_arguments/__init__.py 2.89KB
  200. gpt-neox-main/gpt-neox-main/megatron/optimizers.py 17.69KB
  201. gpt-neox-main/gpt-neox-main/megatron/text_generation_utils.py 33.38KB
  202. gpt-neox-main/gpt-neox-main/megatron/tokenizer/
  203. gpt-neox-main/gpt-neox-main/megatron/tokenizer/tokenizer.py 11.15KB
  204. gpt-neox-main/gpt-neox-main/megatron/tokenizer/train_tokenizer.py 3.89KB
  205. gpt-neox-main/gpt-neox-main/megatron/tokenizer/__init__.py 651B
  206. gpt-neox-main/gpt-neox-main/megatron/training.py 42.11KB
  207. gpt-neox-main/gpt-neox-main/megatron/utils.py 16.87KB
  208. gpt-neox-main/gpt-neox-main/megatron/__init__.py 929B
  209. gpt-neox-main/gpt-neox-main/prepare_data.py 2.28KB
  210. gpt-neox-main/gpt-neox-main/preprocess_data.sh 310B
  211. gpt-neox-main/gpt-neox-main/pretrain.sh 62B
  212. gpt-neox-main/gpt-neox-main/README-MUP.md 1.53KB
  213. gpt-neox-main/gpt-neox-main/README.md 52.62KB
  214. gpt-neox-main/gpt-neox-main/requirements/
  215. gpt-neox-main/gpt-neox-main/requirements/requirements-apex-pip.txt 12B
  216. gpt-neox-main/gpt-neox-main/requirements/requirements-dev.txt 142B
  217. gpt-neox-main/gpt-neox-main/requirements/requirements-flashattention.txt 18B
  218. gpt-neox-main/gpt-neox-main/requirements/requirements-mamba.txt 104B
  219. gpt-neox-main/gpt-neox-main/requirements/requirements-onebitadam.txt 20B
  220. gpt-neox-main/gpt-neox-main/requirements/requirements-s3.txt 25B
  221. gpt-neox-main/gpt-neox-main/requirements/requirements-sparseattention.txt 14B
  222. gpt-neox-main/gpt-neox-main/requirements/requirements-tensorboard.txt 20B
  223. gpt-neox-main/gpt-neox-main/requirements/requirements-wandb.txt 15B
  224. gpt-neox-main/gpt-neox-main/requirements/requirements.txt 395B
  225. gpt-neox-main/gpt-neox-main/tests/
  226. gpt-neox-main/gpt-neox-main/tests/common.py 22.44KB
  227. gpt-neox-main/gpt-neox-main/tests/config/
  228. gpt-neox-main/gpt-neox-main/tests/config/test_setup.yml 1.97KB
  229. gpt-neox-main/gpt-neox-main/tests/conftest.py 3.37KB
  230. gpt-neox-main/gpt-neox-main/tests/cpu_tests/
  231. gpt-neox-main/gpt-neox-main/tests/cpu_tests/action.yml 3.45KB
  232. gpt-neox-main/gpt-neox-main/tests/cpu_tests/docker-compose.yml 506B
  233. gpt-neox-main/gpt-neox-main/tests/data/
  234. gpt-neox-main/gpt-neox-main/tests/data/enwik8_first100.txt 3.28KB
  235. gpt-neox-main/gpt-neox-main/tests/data/hf_cache/
  236. gpt-neox-main/gpt-neox-main/tests/data/hf_cache/tokenizer/
  237. gpt-neox-main/gpt-neox-main/tests/data/hf_cache/tokenizer/gpt2.json 2.01MB
  238. gpt-neox-main/gpt-neox-main/tests/data/sample_prompt.txt 28B
  239. gpt-neox-main/gpt-neox-main/tests/model/
  240. gpt-neox-main/gpt-neox-main/tests/model/test_fused_kernels.py 7.94KB
  241. gpt-neox-main/gpt-neox-main/tests/model/test_model_checkpoint.py 4.06KB
  242. gpt-neox-main/gpt-neox-main/tests/model/test_model_generation.py 3.78KB
  243. gpt-neox-main/gpt-neox-main/tests/model/test_model_instantiation.py 3.85KB
  244. gpt-neox-main/gpt-neox-main/tests/model/test_model_train.py 3.5KB
  245. gpt-neox-main/gpt-neox-main/tests/model/__init__.py 579B
  246. gpt-neox-main/gpt-neox-main/tests/neox_args/
  247. gpt-neox-main/gpt-neox-main/tests/neox_args/test_neoxargs_commandline.py 5.52KB
  248. gpt-neox-main/gpt-neox-main/tests/neox_args/test_neoxargs_implementation.py 914B
  249. gpt-neox-main/gpt-neox-main/tests/neox_args/test_neoxargs_load.py 4.95KB
  250. gpt-neox-main/gpt-neox-main/tests/neox_args/test_neoxargs_usage.py 2.61KB
  251. gpt-neox-main/gpt-neox-main/tests/neox_args/__init__.py 89B
  252. gpt-neox-main/gpt-neox-main/tests/pytest.ini 746B
  253. gpt-neox-main/gpt-neox-main/tests/README.md 1.56KB
  254. gpt-neox-main/gpt-neox-main/tests/test_configs/
  255. gpt-neox-main/gpt-neox-main/tests/test_configs/test_train_base.yml 3.44KB
  256. gpt-neox-main/gpt-neox-main/tests/unit/
  257. gpt-neox-main/gpt-neox-main/tests/unit/test_arguments.py 1.53KB
  258. gpt-neox-main/gpt-neox-main/tests/unit/test_dependencies.py 196B
  259. gpt-neox-main/gpt-neox-main/tests/unit/test_format_conversion_scripts.py 930B
  260. gpt-neox-main/gpt-neox-main/tests/unit/test_launcher_scripts.py 3.84KB
  261. gpt-neox-main/gpt-neox-main/tests/unit/test_tokenizer.py 333B
  262. gpt-neox-main/gpt-neox-main/tests/unit/test_url_accessibility.py 691B
  263. gpt-neox-main/gpt-neox-main/tests/unit/__init__.py
  264. gpt-neox-main/gpt-neox-main/tests/__init__.py
  265. gpt-neox-main/gpt-neox-main/tools/
  266. gpt-neox-main/gpt-neox-main/tools/bash/
  267. gpt-neox-main/gpt-neox-main/tools/bash/kill.sh 16B
  268. gpt-neox-main/gpt-neox-main/tools/bash/killall.sh 55B
  269. gpt-neox-main/gpt-neox-main/tools/bash/README.md 512B
  270. gpt-neox-main/gpt-neox-main/tools/bash/sync.sh 845B
  271. gpt-neox-main/gpt-neox-main/tools/bash/syncdir.sh 905B
  272. gpt-neox-main/gpt-neox-main/tools/bash/sync_cmd.sh 741B
  273. gpt-neox-main/gpt-neox-main/tools/ckpts/
  274. gpt-neox-main/gpt-neox-main/tools/ckpts/convert_hf_to_sequential.py 22.29KB
  275. gpt-neox-main/gpt-neox-main/tools/ckpts/convert_neox_to_hf.py 26.27KB
  276. gpt-neox-main/gpt-neox-main/tools/ckpts/convert_neox_to_mamba_ssm.py 11.63KB
  277. gpt-neox-main/gpt-neox-main/tools/ckpts/convert_raw_llama_weights_to_neox.py 21.61KB
  278. gpt-neox-main/gpt-neox-main/tools/ckpts/inspect_checkpoints.py 11.78KB
  279. gpt-neox-main/gpt-neox-main/tools/ckpts/merge20b.py 9.23KB
  280. gpt-neox-main/gpt-neox-main/tools/ckpts/README.md 5.29KB
  281. gpt-neox-main/gpt-neox-main/tools/ckpts/upload.py 1.51KB
  282. gpt-neox-main/gpt-neox-main/tools/datasets/
  283. gpt-neox-main/gpt-neox-main/tools/datasets/corpora.py 10.54KB
  284. gpt-neox-main/gpt-neox-main/tools/datasets/dataset_token_count.py 876B
  285. gpt-neox-main/gpt-neox-main/tools/datasets/merge_datasets.py 2.26KB
  286. gpt-neox-main/gpt-neox-main/tools/datasets/multinode_prepare_data.sh 2.26KB
  287. gpt-neox-main/gpt-neox-main/tools/datasets/preprocess_data.py 7.56KB
  288. gpt-neox-main/gpt-neox-main/tools/datasets/preprocess_data_with_mask.py 12.3KB
  289. gpt-neox-main/gpt-neox-main/tools/datasets/README.md 5.48KB
  290. gpt-neox-main/gpt-neox-main/tools/README.md 736B
  291. gpt-neox-main/gpt-neox-main/tools/__init__.py
  292. gpt-neox-main/gpt-neox-main/train.py 1.31KB
0评论
提交 加载更多评论
其他资源 lm-dataformat-master.zip
lm-dataformat-master.zip
ZGKJ统计年鉴Excel版(1991-2023年).zip
在数字化时代的浪潮中,数据的重要性日益凸显。对于研究人员、政策制定者以及市场分析师而言,准确、全面的数据是他们分析和决策的基础。《中国科技统计年鉴Excel版(1991-2023年)》正是这样一个宝贵的数据资源,它为了解中国科技进步的脉络提供了一扇窗口。 首先,让我们来了解一下《中国科技统计年鉴》的背景。《中国科技统计年鉴》是由中国国家统计局和科技部联合编纂的年度出版物,自1991年起每年发布一次,至今已有30多年的历史。该年鉴详细记录了中国科技活动的各个方面,包括研发经费投入、科技人才队伍、科研成果产出、技术市场交易等关键指标。 随着信息技术的发展,数据的电子化、表格化成为趋势。《中国科技统计年鉴Excel版》应运而生,它将纸质版的年鉴内容转化为电子表格形式,极大地提高了数据的可访问性和可操作性。用户可以通过Excel软件轻松地检索、筛选、排序和分析数据,这无疑为研究人员提供了极大的便利。
Plain Craft Launcher 2.zip
Plain Craft Launcher 2.zip
window运行pytorch失败
window运行pytorch失败
apex-master.zip
apex-master.zip
萝丽-三代控及接收机源代码
简单易用
Android Launcher应用开发
Android Launcher应用开发Android Launcher应用开发
content_1723521037584.zip
content_1723521037584.zip