Feb 08, 2024 part 1
Jeff Heaton code
with ai-benchmark, I only installed tensorflow and ai-benchmark.
The program from Jeff Heaton's github at
https://github.com/jeffheaton/t81_558_deep_learning/tree/master/install
# What version of Python do you have?
import sys
import tensorflow.keras
import pandas as pd
import sklearn as sk
import tensorflow as tf
print(f"Tensor Flow Version: {tf.__version__}")
print(f"Keras Version: {tensorflow.keras.__version__}")
print()
print(f"Python {sys.version}")
print(f"Pandas {pd.__version__}")
print(f"Scikit-Learn {sk.__version__}")
print("GPU is", "available" if tf.test.is_gpu_available() else "NOT AVAILABLE")
causes error
AttributeError: module 'tensorflow.keras' has no attribute '__version__'
Feb 08, 2024 part 1
Jeff Heaton install, first attempt
Install tensorflow, keras, python in Ubuntu
https://www.youtube.com/watch?v=dj-Jntz-74g
Instructions here:
https://github.com/jeffheaton/t81_558_deep_learning/tree/master/install
Instructions from
https://github.com/jeffheaton/t81_558_deep_learning/blob/master/install/manual_setup.ipynb
Install miniconda
https://docs.anaconda.com/free/miniconda/
mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm -rf ~/miniconda3/miniconda.sh
~/miniconda3/bin/conda init bash
First create a specific environment for conda
cheatsheet at
https://docs.conda.io/projects/conda/en/latest/user-guide/concepts/index.html
https://docs.conda.io/projects/conda/en/latest/user-guide/cheatsheet.html
says:
activate environment
(do this before installing packages)
so perhaps packages will installed in each environment separately?
conda create -y --name tensorflow python=3.6
Then activate this environment before installing packages!
conda activate tensorflow
Install necessary packages
conda install -y jupyter
conda install -y scipy
pip install --exists-action i --upgrade pandas
pip install --exists-action i --upgrade pandas-datareader
pip install --exists-action i --upgrade matplotlib
pip install --exists-action i --upgrade pillow
pip install --exists-action i --upgrade tqdm
pip install --exists-action i --upgrade requests
pip install --exists-action i --upgrade h5py
pip install --exists-action i --upgrade pyyaml
pip install --exists-action i --upgrade tensorflow_hub
pip install --exists-action i --upgrade bayesian-optimization
pip install --exists-action i --upgrade spacy
pip install --exists-action i --upgrade gensim
pip install --exists-action i --upgrade flask
pip install --exists-action i --upgrade boto3
pip install --exists-action i --upgrade gym
pip install --exists-action i --upgrade tensorflow==2.0.0-beta1
pip install --exists-action i --upgrade keras-rl2 --user
conda update -y --all
Run program
# What version of Python do you have?
import sys
import tensorflow.keras
import pandas as pd
import sklearn as sk
import tensorflow as tf
print(f"Tensor Flow Version: {tf.__version__}")
print(f"Keras Version: {tensorflow.keras.__version__}")
print()
print(f"Python {sys.version}")
print(f"Pandas {pd.__version__}")
print(f"Scikit-Learn {sk.__version__}")
print("GPU is", "available" if tf.test.is_gpu_available() else "NOT AVAILABLE")
causes error
Feb 08, 2024 part 2
Jeff Heaton install, second attempt
Install tensorflow, keras, python in Ubuntu
https://www.youtube.com/watch?v=dj-Jntz-74g
Instructions here:
https://github.com/jeffheaton/t81_558_deep_learning/tree/master/install
Instructions from
https://github.com/jeffheaton/t81_558_deep_learning/blob/master/install/tensorflow-install-march-2023.ipynb
curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o
Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
conda install -y jupyter
wget https://raw.githubusercontent.com/jeffheaton/t81_558_deep_learning/master/tools.yml
edit tools.yml
change - cudnn=8.6.0.163 to - cudnn=8.8.0.121
conda env create -f tools.yml --name tensorflow
Pip subprocess error:
ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none)
ERROR: No matching distribution found for tensorflow
failed
Install pips manually:
conda activate tensorflow
pip install --exists-action i --upgrade gym
pip install --exists-action i --upgrade bayesian-optimization
pip install --exists-action i --upgrade kaggle
python3 -m pip install --upgrade
https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.12.0-py3-none-any.whl
from
https://stackoverflow.com/questions/38896424/tensorflow-not-found-using-pip
Run program
# What version of Python do you have?
import sys
import tensorflow.keras
import pandas as pd
import sklearn as sk
import tensorflow as tf
print(f"Tensor Flow Version: {tf.__version__}")
print(f"Keras Version: {tensorflow.keras.__version__}")
print()
print(f"Python {sys.version}")
print(f"Pandas {pd.__version__}")
print(f"Scikit-Learn {sk.__version__}")
print("GPU is", "available" if tf.test.is_gpu_available() else "NOT AVAILABLE")
still causes error
Feb 09, 2024
deactivate conda from auto_activate_base in bash
conda config --set auto_activate_base false
This does not modify .bashrc.
.bashrc before this command:
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/user/miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/home/user/miniconda3/etc/profile.d/conda.sh" ]; then
. "/home/user/miniconda3/etc/profile.d/conda.sh"
else
export PATH="/home/user/miniconda3/bin:$PATH"
fi
fi
unset __conda_setup
# conda initialize
.bashrc after this command:
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/user/miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/home/user/miniconda3/etc/profile.d/conda.sh" ]; then
. "/home/user/miniconda3/etc/profile.d/conda.sh"
else
export PATH="/home/user/miniconda3/bin:$PATH"
fi
fi
unset __conda_setup
# conda initialize
No change, but when I open a new terminal, the prefix (base) is no longer there,
and
python --version
shows a different version.
Feb 10, 2024 part 1
Installed Nvidia GeForce 3050.
Ubuntu 22.04.3 didn't boot.
Booted in recovery mode and installed driver 535.
System boots now but only in low res.
No other resolutions are offered.
https://forums.developer.nvidia.com/t/ubuntu-20-04-lts-nvidia-geforce-rtx-3050-ti-driver-never-loads-or-runs/241269
nvidia-settings
ERROR: NVIDIA driver is not loaded
nvidia-smi
NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest
NVIDIA
driver is installed and running.
Is secure boot/UEFI enabled?
Gigabyte motherboard manual
CSM Support
Enables or disables UEFI CSM (Compatibility Support Module) to support a legacy PC boot process.
Enabled Enables UEFI CSM. (Default)
Disabled Disables UEFI CSM and supports UEFI BIOS boot process only.
try reinstall drivers
https://forums.developer.nvidia.com/t/nvidia-driver-535-54-03-on-ubuntu-22-04-2-lts-and-4070-ti-not-working/259995/3
sudo apt-get remove --purge nvidia-*
Solution
The cause of the problem was that Ubuntu was still configured to run with the previous graphics card
that I replaced with the 3050 card.
Completely reinstalling Ubuntu WITH the 3050 card installed in the PC solved the problem.
A complication arose because the fresh install chose the noveau Nvidia driver, and after reboot, the system
froze.
I rebooted into recovery mode (an option in the grub menu)
and installed the 525 version driver from the "Additional Drivers" program.
After reboot, nvidia-smi and nvidia-settings worked.
Feb 10, 2024 part 2
Some conda experiments
Which version of python is used when auto_activate_base is false?
conda config --set auto_activate_base false
python3 --version
Python 3.10.12
Which version of python is used when auto_activate_base is true?
conda config --set auto_activate_base true
Start new terminal to enact
python3 --version
Python 3.11.5
Specifying the python version for a specific environment
conda create -y --name test01 python=3.6
downloads python-3.6.13
Then activate this environment
conda activate test01
python3 --version
Python 3.6.13 :: Anaconda, Inc.
Feb 10, 2024 part 3
Running jh program again.
still get error
AttributeError: module 'tensorflow.keras' has no attribute '__version__'
Cause may be that tensorflow version is too new!
output from
import tensorflow as tf
print("TensorFlow version:", tf.__version__)
is
TensorFlow version: 2.15.0
"If the TensorFlow version is 2.14 or above, then unfortunately tf.keras.__version__ is undefined."
https://stackoverflow.com/questions/46086030/how-to-check-which-version-of-keras-is-installed
Try
conda install tensorflow=2.1.0
And this resolves the issue.
Feb 14, 2024
Trying to setup a procedure to create an environment where the version of each tools
It seems that when you follow this procedure:
Create a named conda environment
conda create -n my_env python=3.10
and activate this environment
conda activate my_env
and install a python package by using
conda install tensorflow=2.11.0
that this guarantees that in this environment, the correct package version will be used
when starting a program
python3 info.py
Test:
Feb 15, 2024
Observation:
In a new conda environment, named jh_class,
I don't have tensorflow installed in this conda environment.
When I run a program that imports tensorflow,
the program appears to try to load tensorflow from a different location,
revealed by an error message:
import tensorflow.keras
File "/home/user/.local/lib/python3.10/site-packages/tensorflow/__init__.py", line 40, in
Even after installing tensorflow version 2.12.0 in this conda environment,
running the program
python3 -c 'import tensorflow as tf; print("Tensor Flow Version:"); print(tf.__version__)'
results in the output
Tensor Flow Version:
2.15.0
How do I make sure that the package version that I want is being used?
When I remove the folder
/home/user/.local/lib/python3.10/site-packages
and run the program again, then the output is
Tensor Flow Version:
2.12.0
Feb 16, 2024 part 2
The backprop example
t81_558_class_04_4_backprop
works in the environment
jh_tensorflow
where the versions are
Tensor Flow Version: 2.1.0
Python 3.7.16 (default, Jan 17 2023, 22:20:44) [GCC 11.2.0]
Pandas 1.3.5
Scikit-Learn 1.0.2
The backprop example
t81_558_class_04_4_backprop
fails in the environment
jh_class
where the versions are
Tensor Flow Version: 2.12.0
Python 3.10.13 (main, Sep 11 2023, 13:21:10) [GCC 11.2.0]
Pandas 2.1.4
Scikit-Learn 1.3.0
and in the environment jh_class_02
where the versions are
Tensor Flow Version: 2.15.0
Python 3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:36:39) [GCC 12.3.0]
Pandas 2.2.0
Scikit-Learn 1.4.1.post1
with the error message
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float).
Feb 16, 2024 part 3
The huggingface example
t81_558_class_11_01_huggingface
downloads several large models
No model was supplied, defaulted to distilbert-base-uncased-finetuned-sst-2-english and revision af0f99b
(https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english).
And
No model was supplied, defaulted to dbmdz/bert-large-cased-finetuned-conll03-english and revision f2482bf
(https://huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english).
How can we download these ahead of running the program,
and point the program to the local storage?
https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading
Feb 17, 2024 part 1
Decided to stick with tensorflow 2.15, the latest version,
and to modify the JH code instead.
Fix for the t81_558_class_11_01_huggingface error:
Replace
# Convert to numpy - Classification
x_columns = df.columns.drop('age').drop('id')
x = df[x_columns].values
y = df['age'].values
with
# Convert to numpy - Classification
x_columns = df.columns.drop('age').drop('id')
x = df[x_columns].values
y = df['age'].values
x = x.astype('float32')
y = y.astype('float32')
(Based on Brave AI answer)
Feb 17, 2024 part 2
GPU support with tensorflow
create conda environment
conda create -n test01 python=3.10
activate
conda activate test01
Install CUDA-enabled tensorflow
conda install tensorflow[and-cuda]
install CUDA
CUDA/index.html
Install cuDNN
CUDnn/index.html
Verify that GPU is available to tensorflow
python3 -c 'import tensorflow as tf; print("Num GPUs Available: ",
len(tf.config.list_physical_devices("GPU")))'
2024-02-17
GPU is not seen
output is much shorter than on working system
2024-02-17 15:15:30.117592: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is
optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with
the appropriate compiler flags.
[]
in Vscode, got error
bash: export: `:/usr/local/cuda-12.3/lib64': not a valid identifier
removed space from
export LD_LIBRARY_PATH=/usr/local/cuda-12.3/lib64 ${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
https://www.reddit.com/r/JetsonNano/comments/135mlue/bash_export_usrlocalcudalib64_is_not_a_valid/?rdt=65230
now library path command givs no error, but GPU still not seen.
observation:
conda list
shows no cuda packages.
However,
conda create -n test02 python=3.10
activate
conda activate test02
Install CUDA-enabled tensorflow
conda install -c conda-forge tensorflow=2.15.0
Now output of
python3 -c 'import tensorflow as tf; print("Num GPUs Available: ",
len(tf.config.list_physical_devices("GPU")))'
is
2024-02-17 15:26:36.755517: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register
cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-02-17 15:26:36.755559: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register
cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-02-17 15:26:36.756412: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to
register
cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-02-17 15:26:36.761468: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is
optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with
the appropriate compiler flags.
2024-02-17 15:26:37.843365: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful
NUMA
node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node
zero. See more at
https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2024-02-17 15:26:37.938638: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful
NUMA
node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node
zero. See more at
https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2024-02-17 15:26:37.938885: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful
NUMA
node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node
zero. See more at
https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
Num GPUs Available: 1
and
conda list shows cuda packages.
Maybe order of installation?
Install CUDnn first and then tensorflow? Or is it tensorflow 2.15?
Also, note that I installed tensorflow without the [cuda] stuff,
and it still works
Feb 28, 2024
Keras Example: Simple TensorFlow Regression: MPG
from Jeff Heaton class
mpg dataset is in file auto-mpg.csv:
| mpg |
cylinders |
displacement |
horsepower |
weight |
acceleration |
year |
origin |
name |
| 18 |
8 |
307 |
130 |
3504 |
12 |
70 |
1 |
chevrolet chevelle malibu |
| 15 |
8 |
350 |
165 |
3693 |
11.5 |
70 |
1 |
buick skylark 320 |
| 18 |
8 |
318 |
150 |
3436 |
11 |
70 |
1 |
plymouth satellite |
The data are read into a Pandas DataFrame named df.
The name column is extraced into the variable named cars.
The dataframe is copied into two numpy arrays:
- x containing 'cylinders', 'displacement', 'horsepower', 'weight', 'acceleration', 'year', 'origin'
- y containing mpg values
A neural network model is created
The neural network parameters are set to
model.compile(loss='mean_squared_error', optimizer='adam')
Then the mpg data are fitted to the
model.fit(x,y,verbose=2,epochs=100)
Feb 29, 2024
Other keras regression examples
"[AI] Understanding the parameters of model.compile() and model.fit() in Tensorflow Keras"
https://www.youtube.com/watch?v=VCrIBJ9rkmg
Badri Adhikari
Keras regression example
Goal: take blood pressure as input and generate age as prediction output.
Does not work with these input data
Input data:
| age |
blood pressure |
| 10 |
82 |
| 22 |
85 |
| 31 |
89 |
| 45 |
92 |
| 52 |
95 |
| 61 |
97 |
| 72 |
98 |
Code:
model = Sequential()
model.add(Dense(1, input_dim=1, activation='linear'))
model.compile(loss='mse', optimizer='rmsprop',metrics='mae')
model.fit(bp,age,verbose=0,epochs=1000)
Output:
1. blood pressure: 82, age: 10, predicted age: [-28.06324]
2. blood pressure: 83, age: 15, predicted age: [-28.41769]
3. blood pressure: 85, age: 22, predicted age: [-29.12659]
4. blood pressure: 89, age: 31, predicted age: [-30.544395]
5. blood pressure: 92, age: 45, predicted age: [-31.60775]
6. blood pressure: 93, age: 47, predicted age: [-31.962204]
7. blood pressure: 95, age: 52, predicted age: [-32.671104]
more regression examples
"Linear regression with two input variables using tensorflow keras"
https://www.youtube.com/watch?v=IOaif62O06k
Badri Adhikari
