Skip to content

Transformers

Install packages

1
2
3
4
5
6
7
!uv pip install -q \
    transformers==4.57.3 \
    bertviz==1.4.1 \
    xformers==0.0.33.post2 \
    evaluate==0.4.6 \
    numpy==2.3.2 \
    matplotlib==3.10.8

Import packages

1
2
3
4
5
6
import matplotlib.pyplot as plt
import numpy as np
from bertviz.neuron_view import show
from bertviz.transformers_neuron_view import BertModel as BertModelViz
from bertviz.transformers_neuron_view import BertTokenizer as BertTokenizerViz
from transformers import BertModel, BertTokenizer, pipeline

Define model and tokenizer

1
2
3
4
model_name = "bert-base-uncased"

tokenizer = BertTokenizer.from_pretrained(model_name)
model = BertModel.from_pretrained(model_name)

Tokenize a sentence

1
2
tokenized = tokenizer("I read a good novel")
tokenized

{'input_ids': [101, 1045, 3191, 1037, 2204, 3117, 102], 'token_type_ids': [0, 0, 0, 0, 0, 0, 0], 'attention_mask': [1, 1, 1, 1, 1, 1, 1]}

Inspect the tokens

1
2
tokens = tokenizer.convert_ids_to_tokens(tokenized["input_ids"])
tokens

['[CLS]', 'i', 'read', 'a', 'good', 'novel', '[SEP]']

Positional Encoding

Define encode position function

1
2
3
4
5
6
7
8
def encode_positions(num_tokens, depth, n=10000):
    positional_matrix = np.zeros((num_tokens, depth))
    for row in range(num_tokens):
        for col in np.arange(int(depth / 2)):
            denominator = np.power(n, 2 * col / depth)
            positional_matrix[row, 2 * col] = np.sin(row / denominator)
            positional_matrix[row, 2 * col + 1] = np.cos(row / denominator)
    return positional_matrix

Plot positional matrix

1
2
3
4
positional_matrix = encode_positions(50, 256)
fig = plt.matshow(positional_matrix)
plt.gcf().colorbar(fig)
plt.show()
output_15_0.png

Self-attention

1
2
3
tokenizer_viz = BertTokenizerViz.from_pretrained(model_name)
model_viz = BertModelViz.from_pretrained(model_name)
show(model_viz, "bert", tokenizer_viz, "I read a good novel.", head=11)

GPT 2

1
2
generator = pipeline("text-generation", model="gpt2")
generator("I read a good novel.", max_length=30, num_return_sequences=5)

Device set to use cpu

Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.

Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.

Both `max_new_tokens` (=256) and `max_length`(=30) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)

[{'generated_text': "I read a good novel. I have a lot of friends who are writers, and I read about writers. I am a writer. That is what I do.\n\nI have a lot of friends who are writers, and I read about writers. I am a writer. That is what I do. I read about authors. I am a writer. I am a writer. I am a writer. I am a writer. I am a writer. I am a writer.\n\nI can't say my name, because I feel like I am not the type of person to say it. I can't say my name, because I feel like I am not the type of person to give my name out and shout it out. I can't say my name, because I feel like I am not the type of person to do it.\n\nI really don't want to be around anyone who is not a writer. I want to be around people who are not writers. I want to be around people who are not writers. I want to be around people who are not writers. I want to be around people who are not writers. I want to be around people who are not writers. I want to be around people who are not writers.\n\nI am a writer. I"},
 {'generated_text': 'I read a good novel. I have my doubts. I\'m not saying it\'s bad, but it\'s kind of the same book as I was reading back in college. I don\'t think it\'s bad. I\'m sure it\'s OK sometimes, but it\'s not like I\'m giving this book 4 stars. It\'s like I might have written it wrong. At least, that\'s what I was told.\n\nWhat is your opinion on the criticism of your book?\n\nI feel like this story is really a good example of how you can bring something to life and get it to you. It\'s also about how you can be different in your own way. This is a new kind of book, right? And that\'s what I love about it so much. It\'s about the way you can be different in your own way and how you can be different in your own way.\n\nWhat about the criticism you receive?\n\nI think the biggest thing I get is people who say, "Oh, you can\'t write a book like this, you\'re better off writing a book like this." I don\'t think that\'s true. I don\'t think that\'s true. I think you need to write a novel that is about a person who is in an abusive relationship'},
 {'generated_text': 'I read a good novel. It had a very nice story of a beautiful young girl in a terrible place. She was kidnapped and she was brought to the United States by her brother and her whole family. Then she was brought back in the United States and killed by her sister, who was the head of a big corporation. She left for England and she went to India, where she became a nun and spent a lot of time studying history and philosophy. She was a pretty young woman. Then she got her job as a translator, and she was very successful. We were very much on the same side. We were making quite a lot of money, and I think all of us were very happy.\n\nThe main character in the book, G.G., was born in England in 1868 and he was very much the grandson of the late Thomas, who died a young man. He was a beautiful young man, and he was very intelligent. He was very ambitious, and he was very good at what he did. He was a good writer. He was also very clever, and he was very good at making jokes.\n\nHe was very fond of playing the violin, and he also played the piano. He was a very good player, and he played both the piano and the trumpet'},
 {'generated_text': "I read a good novel.\n\nI have a girlfriend who has a husband who has children.\n\nI've been married for four years.\n\nI'm getting married in April.\n\nI have a lot of other things in my life.\n\nI'm doing well.\n\nI am going to make a big mistake.\n\nI'm going to be very selfish.\n\nI'm going to make the next mistake.\n\nI'm going to take a life without me.\n\nI'm going to be honest.\n\nI'm going to be honest.\n\nI'm going to be honest."},
 {'generated_text': 'I read a good novel. I do not pretend to know what the other person is thinking, but I know my own mind. I am only a little bit of a writer. I wrote this book the same way that I wrote the book I did the previous year.\n\nA few years ago, my husband and I wrote a book called "Why I Don\'t Like Politics." We wrote about Trump, and it was a good book. I said, "I think Trump is a fraud." And I said, "No, he is not a fraud." I said, "Well, he is my friend, and I know he is the person who is going to get elected." So I said, "I understand. You said that. Don\'t you understand?" And he said, "No, I don\'t."\n\nThe book was called "Why Trump is a Fraud." It was called "Why I Don\'t Like Bernie Sanders." It was called "Why I Don\'t Like Trump." It was called "What\'s it like to be a Trump supporter?" I said, "What\'s it like to be a Trump supporter?" and the book is called "Why Trump is a Fraud." It was called "Why I Don\'t Like Bernie Sanders." It was called "What\'s it like'}]