diff --git a/README.md b/README.md index 9debd17..e78e3ee 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@ # minGPT + ![mingpt](mingpt.jpg) A PyTorch re-implementation of [GPT](https://github.com/openai/gpt-2), both training and inference. minGPT tries to be small, clean, interpretable and educational, as most of the currently available GPT model implementations can a bit sprawling. GPT is not a complicated model and this implementation is appropriately about 300 lines of code (see [mingpt/model.py](mingpt/model.py)). All that's going on is that a sequence of indices feeds into a [Transformer](https://arxiv.org/abs/1706.03762), and a probability distribution over the next index in the sequence comes out. The majority of the complexity is just being clever with batching (both across examples and over sequence length) for efficiency. diff --git a/mingpt/model.py b/mingpt/model.py index 83ee22d..50e02ed 100644 --- a/mingpt/model.py +++ b/mingpt/model.py @@ -8,8 +8,13 @@ https://github.com/huggingface/transformers/blob/main/src/transformers/models/gpt2/modeling_gpt2.py """ -import math +## changing for main branch + +## changing for merge branch + +import math +# change 2 import torch import torch.nn as nn from torch.nn import functional as F