bert-trump-tweet-voo

This model is a fine-tuned version of distilbert-base-uncased on Trump's tweets in his first presidential term vs S&P 500 performance (VOO), with a buy or sell classification based on forward-looking market performance of more than an 0.5% movement in the market.

Model description

Repository will be made available at https://github.com/haplesshero13/trump-voo

Intended uses & limitations

This is not intended to be financial advice. Investing is subject to market risks. Investors acknowledge and accept the potential loss of some or all of an investment's value.

Training and evaluation data

The training set can be found at https://www.kaggle.com/datasets/codebreaker619/donald-trump-tweets-dataset/data?select=tweets.csv and yfinance.

The model was evaluated on Truth Social posts from Term 2 using https://trumpstruth.org/feed and yfinance data.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

• global_step: 939
• grad_norm: 2.46255
• learning_rate: 0.0000040469
• loss: 1.04
• total_flos: 7960902674415616

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.2.0
• Tokenizers 0.22.1