home scroll AI Learning

Jeff Heaton Class

jh

Setting up the Python Environments

Conda allows us to set up a defined environment with specific versions of the python version as well as of the packages.

conda environment: jh_class

This environment works with most of the modules in the class.
conda create -n jh_class python=3.10
Then activate this environment before installing packages!
conda activate jh_class
Install specific packages:
conda install -c conda-forge tensorflow=2.15.0 -y
conda install -c conda-forge pandas=2.2.0 -y
conda install -c conda-forge scikit-learn=1.6.1 -y
conda install -c conda-forge matplotlib=3.8.3 -y
conda install -c conda-forge ipython=8.21.0 -y
conda install -c conda-forge transformers=4.37.2 -y
conda install -c conda-forge jupyter=1.0.0 -y
conda install -c conda-forge sentencepiece=0.1.99 -y
conda install -c conda-forge click=8.0.4 -y
Install packages (without version number):
conda install -c conda-forge imageio -y
conda install -c conda-forge keras-preprocessing -y
conda install -c conda-forge pydot -y
conda install -c conda-forge tensorflow-datasets -y
conda install -c conda-forge bayesian-optimization -y

This command lists the package versions that are installed in the currently active environment:
conda list
Result:
Computer without GPU
Computer with GPU

conda environment: stylegan3

Module 7 Part 7.2

The class module of class 07_2 uses stylegan2, however the environment did not work with the RTX 3090 card.

Instead, I am using the stylegan3 examples from
https://github.com/NVlabs/stylegan3
Details on the stylgan3 environment are on the computer vision page:
Computer Vision

conda environment: tabgan

Module 7 Part 7.5

Adding tabgan to jh_class will corrupt the jh_class environment such that the class programs will not run any longer,

It is better to create a separate conda environment for the tabgan module 7.5.
https://github.com/Diyago/Tabular-data-generation

To run the included example:

Create conda environment
conda create -n mytabgan python=3.10
Activate conda environment
conda activate mytabgan
Install packages
conda install -y pandas
pip install tabgan

conda environment: ensemble

Module 8 Part 8.2 - Biological Response with Neural Network - Neural Network Ensemble

class_08_2_keras_ensembles_bio_blend.py
conda create -n ensemble python=3.7
conda install -c conda-forge numpy -y
conda install -c conda-forge pandas -y
conda install -c conda-forge tensorflow -y
pip install scikit-learn

Class

To continue the class:
Open terminal
cd ~/github/learn/ai/jheaton/t81_558_deep_learning
conda activate jh_class
jupyter notebook

t81_558_class_01_1_overview.ipynb..........................| 2024-02-12 t81_558_class_01_2_intro_python.ipynb......................| 2024-02-12 t81_558_class_01_3_python_collections.ipynb................| 2024-02-13 t81_558_class_01_4_python_files.ipynb......................| 2024-02-20 t81_558_class_01_5_python_functional.ipynb.................| 2024-02-21 t81_558_class_02_1_python_pandas.ipynb.....................| 2024-02-22 t81_558_class_02_2_pandas_cat.ipynb........................| 2024-02-24 t81_558_class_02_3_pandas_grouping.ipynb...................| 2024-02-25 t81_558_class_02_4_pandas_functional.ipynb.................| 2024-02-25 t81_558_class_02_5_pandas_features.ipynb...................| 2024-02-26 t81_558_class_03_1_neural_net.ipynb........................| 2024-02-26 t81_558_class_03_2_keras.ipynb.............................| 2024-03-06 t81_558_class_03_3_save_load.ipynb.........................| 2024-03-06 t81_558_class_03_4_early_stop.ipynb........................| 2024-03-06 t81_558_class_03_5_weights.ipynb...........................| 2024-03-07 t81_558_class_04_1_feature_encode.ipynb....................| 2024-03-08 t81_558_class_04_2_multi_class.ipynb.......................| 2024-03-12 t81_558_class_04_3_regression.ipynb........................| 2024-03-13 t81_558_class_04_4_backprop.ipynb..........................| 2024-07-12 t81_558_class_04_5_rmse_logloss.ipynb......................| 2024-07-12 t81_558_class_05_1_reg_ridge_lasso.ipynb...................| 2024-09-19 t81_558_class_05_2_kfold.ipynb.............................| 2024-10-27 t81_558_class_05_3_keras_l1_l2.ipynb.......................| 2024-11-18 t81_558_class_05_4_dropout.ipynb...........................| 2024-11-19 t81_558_class_05_5_bootstrap.ipynb.........................| 2024-12-06 t81_558_class_06_1_python_images.ipynb.....................| 2024-12-08 t81_558_class_06_2_cnn.ipynb...............................| 2024-12-27 t81_558_class_06_3_resnet.ipynb............................| 2024-12-31 t81_558_class_06_4_keras_images.ipynb......................| 2025-01-13 t81_558_class_06_5_yolo.ipynb..............................| 2025-01-14 t81_558_class_07_1_gan_intro...............................| 2025-01-18 t81_558_class_07_2_train_gan.ipynb.........................| 2025-01-24 t81_558_class_07_3_latent_vector.ipynb.....................| 2025-01-24 t81_558_class_07_4_deoldify.ipynb..........................| 2025-01-25 t81_558_class_07_5_tabular_synthetic.ipynb.................| 2025-01-28 t81_558_class_08_1_kaggle_intro.ipynb......................| 2025-04-21 t81_558_class_08_2_keras_ensembles.ipynb...................| 2025-05-23 t81_558_class_08_3_keras_hyperparameters.ipynb.............| 2025-05-24 t81_558_class_08_4_bayesian_hyperparameter_opt.ipynb.......| t81_558_class_08_5_kaggle_project.ipynb....................| t81_558_class_09_1_keras_transfer.ipynb....................| t81_558_class_09_2_keras_xfer_cv.ipynb.....................| t81_558_class_09_3_transfer_nlp.ipynb......................| t81_558_class_09_4_facial_points.ipynb.....................| t81_558_class_09_5_style_transfer.ipynb....................| t81_558_class_10_1_timeseries.ipynb........................| t81_558_class_10_2_lstm.ipynb..............................| t81_558_class_10_3_text_generation.ipynb...................| t81_558_class_10_4_intro_transformers.ipynb................| t81_558_class_10_5_keras_transformers.ipynb................| t81_558_class_11_01_huggingface.ipynb......................| t81_558_class_11_02_tokenizers.ipynb.......................| t81_558_class_11_03_hf_datasets.ipynb......................| t81_558_class_11_04_hf_train.ipynb.........................| t81_558_class_11_05_embedding.ipynb........................| t81_558_class_12_01_ai_gym.ipynb...........................| t81_558_class_12_02_qlearningreinforcement.ipynb...........| t81_558_class_12_03_keras_reinforce.ipynb..................| t81_558_class_12_04_atari.ipynb............................| t81_558_class_12_05_apply_rl.ipynb.........................| t81_558_class_13_01_flask.ipynb............................| t81_558_class_13_02_checkpoint.ipynb.......................| t81_558_class_13_03_web.ipynb..............................| t81_558_class_13_04_retrain.ipynb..........................| t81_558_class_13_05_tpu.ipynb..............................| t81_558_class_14_01_automl.ipynb...........................| t81_558_class_14_02_auto_encode.ipynb......................| t81_558_class_14_03_anomaly.ipynb..........................| t81_558_class_14_04_ids_kdd99.ipynb........................| t81_558_class_14_05_new_tech.ipynb.........................|

Software version issues

Installation in February 2024 installs tensorflow 2.15.0.
t81_558_class_01_1_overview.ipynb calls tensorflow.keras.__version
However, this attribute was deprecated after tensorflow 2.13.0,
so leads to an error.
https://stackoverflow.com/questions/46086030/how-to-check-which-version-of-keras-is-installed
This can be resolved by running
conda install tensorflow=2.1.0
in the according environment in which this notebook is run.

Other

The syntax for version numbers in pip is
pip install tensorflow==2.15.0
but we don't want to install anything outside of conda,
so that we know that we are using the intended versions of our packages.

Create PDF from jupyter notebook

Using nbconvert: export to PDF

https://saturncloud.io/blog/how-to-export-jupyter-notebook-by-vscode-in-pdf-format/
requires TeX
https://nbconvert.readthedocs.io/en/latest/install.html#installing-tex
pip install nbconvert PyPDF2

- Open the notebook in VS Code or Codium
- press CTRL-SHIFT-P
- type Export
- choose Jupyter: Export to PDF

Using nbconvert: export to Html

Does not require TeX, but sometimes fails

- Open the notebook in VS Code or Codium
- press CTRL-SHIFT-P
- type Export
- choose Jupyter: Export to Html
- Open Html file in browser
- Print to PDF


Follow Me

discord