Notes From Class

Week 01: January 24 and 26, 2022.

Topic (2022-01-24): Welcome to ENCE 688P ...
Topic (2022-01-24): Data and Information Management in the Built Environment (Part 1): (pdf) (zoom video)
Topic (2022-01-26): Data and Information Management in the Built Environment (Part 2): (pdf) (zoom video)
Topic (2022-01-26): Data and Information Management in the Built Environment (Part 3): (pdf) (zoom video)
Download: Data and Information Management in the Built Environment (Parts 1-2-3) (pdf) (Version: 2022-01-26)

Week 02: January 31 and February 2, 2022.

Topic (2022-01-31): Model-based Systems Engineering --> Semantics + Data Mining (Part 1): (pdf) (zoom video)
Topic (2022-02-02): Model-based Systems Engineering --> Semantics + Data Mining (Part 2): (pdf) (zoom video)
Topic (2022-02-02): Model-based Systems Engineering --> Semantics + Data Mining (Part 3): (pdf) (zoom video)
Minor Update: Model-based Systems Engineering --> Semantics + Data Mining (Parts 1-2-3) (pdf)(Version: 2022-01-31)
Reading. Wang W., Yang S., Stanley H.E., Gao J.,
Local Floods induce Large-Scale Abrupt Failures of Road Networks (pdf),
Nature Communications, May, 2019.
Note: Saini was a PhD student here at UMD during the 2000-2006 time frame.
Reading. Bello J.P., Silva C., Dubios R.L., Arora A., Salamon J., Mydlarz C., and Doraiswamy H.,
SONYC: A System for Monitoring, Analyzing and Mitigating Urban Noise Polution (pdf),
Contributions of the ACM, Vol. 62, No. 2, February 2019, pp. 68-77.

Week 03: February 7 and 9, 2022.

Topic (2022-02-07): Engineering Software Development (Part 1) (pdf) (zoom video) (coming soon).
Topic (2022-02-07): Engineering Software Development (Part 2) (pdf) (zoom video) (coming soon).
Topic (2022-02-09): Introduction to Java (History and Buzzwords) (pdf ) (zoom video)
Topic (2022-02-09): Introduction to Java (First Program) (pdf) (zoom video)
Download: Engineering Software Development (Parts 1-2) (pdf) (Version 2021-02-05)
Click here to download a zip file of the Python code (Version: 2022-02-08).
Unpack, then look at the README.txt file.

This version contains scripts for:
- Basic stuff: variables, math functions, loops, strings, methods, arrays, objects.
- Formatting string output.
- Reading input from CSV and JSON files.
- Working with collections: lists, sets, and dictionaries.
- Working with NumPy: matrix transformations and matrix linear algebra.
- Working with Pandas: series and dataframes.
- Creating two- and three-dimensional plots.
- Computing and plotting sigmoid functions.
- Working with dates and time, and (basic) time series.
- Creating objects and working with inheritance mechanisms.
UPDATE: Click here to download java-code-basics (Version: 2022-02-23).

The updated version contains targets for:
- Basic stuff: variables, math functions , loops, strings, methods, arrays, objects.
- Using loops to compute series summation.
- Working with enumerated data types.
- Demonstrating the limitations of numerical computations.
- Working with inheritance mechanisms, abstract classes and interfaces.
- Modeling two-dimensional vectors and line segments.
- A simplified implementation of matrices and matrix operations.
- Working with scanners; reading input from CSV files.
- Creating two-dimensional charts with JFreeChart.
- Computing and plotting sigmoid functions.
- Working with time,
Click here to download a zip file of java-code-ant (Version: 2021-09-20).
Unpack, then look inside the folder java-code-basics. Apache Ant uses the file build.xml.
For a list of the programs, open a terminal window and type: ant -p
Homework 1: Homework 1
I suggest that you submit your homework (source code + output) as a zip file.
Due: February 21.

Week 04: February 14 and 16, 2022.

Topic (2022-02-14): The Java Language (Part 1) (Basic Stuff) (pdf) (zoom video)
Topic (2022-02-14): The Java Language (Part 2) (Packages and Methods) (pdf) (zoom video)
Topic (2022-02-16): The Java Language (Part 3) (Arrays) (pdf) (zoom video)
Download: The Java Language (Parts 1-2-3) (pdf) (Version: 2020-09-28)
Topic (2022-02-16): Java Productivity Tools (Apache Ant) (pdf) (zoom video)

Week 05: February 21 and 23, 2022.

Topic (2022-02-21): Working with Classes and Objects (Part 1) (Objects, Relationship among Classes) (pdf) (zoom video)
Topic (2022-02-21): Working with Classes and Objects (Part 2) (Association Relationships) (pdf) (zoom video)
Topic (2022-02-21): Working with Classes and Objects (Part 3) (Composition and Applications) (pdf) (zoom video)
Download: Working with Classes and Objects (Parts 1-2-3) (pdf) (Version: 2020-10-10)
Topic (2022-02-23): Abstract Classes and Interfaces (Part 1) (Abstract Classes) (pdf) (zoom video)
Topic (2022-02-23): Abstract Classes and Interfaces (Part 2) (Interfaces) (pdf) (zoom video)
Topic (2022-02-23): Abstract Classes and Interfaces (Part 3) (Applications) (pdf) (zoom video)
Download: Abstract Classes and Interfaces (Parts 1-2-3) (pdf)
Homework 2: Homework 2
I suggest that you submit your homework (source code + output) as a zip file.
Due: March 07.

Week 06: February 28 and March 2, 2022.

Topic (2022-02-28): Java Collections (pdf) (zoom video).
Topic (2022-02-28): Tutorial from IBM: Java Collections Framework
This tutorial is a bit old, but has very nice examples ...
UPDATE: Click here to download a zip file of java-code-collections (Version: 2022-04-05).

Either load into your favorite IDE (Eclipse) or open a terminal window and type: ant -p

You will find about twenty test programs covering use of arraylists, hashmaps, linked hashmaps,
hashsets, symbol tables, and programming for association relationships.
Click here to download a zip file of java-code-design-patterns (Version: 2021-02-25).

You will find target programs for the adapter, bridge, builder, chain, composite, factory, mediator, model-view-controller, observer, state, strategy and visitor design patterns.
A few screenshots from recent work:

Left: BWI Airport.
Right: NYC subway system and coastline.
Interview: Lex Fridman hosts Travis Oliphant, creator of Python packages for scientific computation (e.g., NumPy, SciPy)
and distribution platforms (e.g., Anaconda).
Presentation Slides. Maria Coelho, Distributed System Behavior Modeling of Urban Systems with Ontologies,
Rules and Many-to-Many Association Relationships (pdf),
ICONS 2017, Venice, Italy, April 2017.

Week 07: March 7 and 9, 2022.

Topic (2022-03-07): Introduction to Machine Learning (Part 1) (Introduction) (pdf) (zoom video)
Topic (2022-03-07): Introduction to Machine Learning (Part 2) (Machine Learning Capabilities) (pdf) (zoom video)
Topic (2022-03-09): Introduction to Machine Learning (Part 3) (Taxonomy of ML Problems) (pdf) (zoom video)
Topic (2022-03-09): Introduction to Machine Learning (Part 4) (Types of ML Systems) (pdf) (zoom video)
Topic (2022-03-09): Introduction to Machine Learning (Part 5) (Urban Applications) (pdf) (zoom video)
Download: Introduction to Machine Learning (Parts 1-5) (pdf) (Version 2022-03-01).

Week 08: March 14 and 16, 2022.

Topic (2022-03-14): Data Mining (Part 1) (Introduction) (pdf) (zoom video)
Topic (2022-03-14): Data Mining (Part 2) (Entropy, Probability, Information Gain) (pdf) (zoom video)
Topic (2022-03-14): Data Mining (Part 3) (Information Gain in Decision Trees) (pdf) (zoom video)
Topic (2022-03-14): Data Mining (Part 4) (Ensemble Learning) (pdf) (zoom video)
Topic (2022-03-16): Data Mining (Part 5) (Working with Weka, Examples) (pdf) (zoom video)
Download. Introduction to Data Mining (Parts 1-5) (pdf) (Version 2021-10-16).

Week 09: March 21 and 23, 2022.

Spring Break!

Week 10: March 28 and 30, 2022.

Ian Witten's Presentation Slides on Working with WEKA:
Data Mining With Weka: Class 1 (Introduction) (pdf).
Data Mining With Weka: Class 2 (Be a Classifier) (pdf).
Data Mining With Weka: Class 3 (Simplicity First) (pdf).
Data Mining With Weka: Class 4 (Classification Boundaries) (pdf).
Data Mining With Weka: Class 5 (Putting it Together) (pdf).
A zip file of Weka code and examples will be distributed this week.
Check your e-mail for a message from Dropbox.
Reading. Ledoux H., Ohori K., Kumar K., Dukai B., Labetski A., ad Vitalis S.,
CityJSON: A compact and easy-to-use encoding of the CityGML Data Model (pdf),
arXiv:1902.09155v3, 2020.

Week 11: April 4 and 6, 2022.

Introduction to ML with Neural Networks:

Reading. Lippmann R.P., An Introduction to Computing with Neural Nets (pdf),
IEEE ASSP Magazine, April 1987. See, in particular, Figures 1, 3, 12 and 14.
Reading. Bhiksha R., Introduction to Neural Networks (pdf),
Lisbon Machine Learning School, June, 2018. Fantastic introduction to neural networks and their capabilities.
Reading. Artificial networks learn to smell like the brain,
MIT News, October 2021.
Topic (2022-04-04): Neural Networks I (Part 1) (Perceptron Models) (pdf) (zoom video)
Topic (2022-04-06): Neural Networks I (Part 2) (Training Perceptron Models) (pdf) (zoom video)
Download. Click here to download Neural Networks I (Parts 1-2) (Version: 2022-05-27).
Location of Software Examples:
Example 1: Modeling an OR Boolean Gate.
Python + NumPy. See: python-code.d/neural/TestNeural-BooleanORGate.py

Example 2: Modeling an OR Boolean Gate.
Deeplearning4J. See: src/nn/basic/BooleanGateOR.java
Homework 3: The purposes of this homework are to learn:
- How the Java collections can be used in data models, and
- How to export a data model to Weka (using the arff file format).
We will load the latter into Weka and then look for classification relationships among the various attributes.
Getting Started: Download an updated version of java-code-collections (Version: 2022-04-05).
Unzip and then look at the collection of files in /src/homework03/

There are two new test programs:
- Target: ant homework03a.
  Creates a weather data model, and then imports a data file (see data/seattle-weather.csv) ontaining four years of daily weather measurements (e.g., precipitation, temperature) in Seattle.
- Target: ant homework03b.
  Creates a passenger data model, and then imports a data file (se data/titanic.csv) containing information on 887 passengers who were on the Titantic.
Pay particular attention to how data is stored in WeatherDataItem.java vs PassengerDataItem.java.
We can discuss these differences in class.
Things to do:
- Fill in the missing details of the method exportARFF() in WeatherDataModel.java.
  
  Since this data file will be used for predicting various kinds of weather conditions, I suggest that you split the date into three separate columns for day, month and year.
- Load the exported data file into Weka and see if you can get it to predict when it is most likely to rain (or fog or snow) in Seattle?
- Extend the functionality of TestPassengerDataModel.java so that you can query a specific passenger on the Titanic, and then print his/her details.
- Fill in the missing details of the method exportARFF() in PassengerDataModel.java.
- Load the exported data file into Weka and see if you can get it to identify parameters (e.g., age?) that determined whether or not a passenger survived.
- Hand in a zip of your working code and a brief writeup of any predictions that were discovered.
Notes:
- You can find a disussion on the ARFF file format in Section 2.4 of the Data Mining Text by Witten et al.
- Also see the class home page for links to a tutorial on the ARFF file format
  and an online site that automatically converts CSV files to ARFF.
  You can use the latter to check your computations.
- As a follow-up to our discussion in office hours on Monday (April 18) on lists, filters, and maps,
  I am posting a slightly updated version of java-code-collections.
  The download will unzip to a folder called java-code-collections2022-04 --- the new temporary
  name should avoid anyone accidentally erasing their homework.
  To Run: ant filter01, etc ... my experiments with titanic datamodel are src/filter04/ ...
Due Date: May 1, 2022

Week 12: April 11 and 13, 2022.

Topic (2022-04-11): Neural Networks II (Part 1) (Introduction to MLP) (pdf) (zoom video)
Topic (2022-04-11): Neural Networks II (Part 2) (One Hidden Layer) (pdf) (zoom video)
Topic (2022-04-13): Neural Networks II (Part 3) (Two Hidden Layers) (pdf) (zoom video)
Download: Click here to download Neural Networks II (Parts 1-2-3) (Version: 2021-11-08).
Location of Software Examples:
Example 3: Modeling an XOR Boolean Gate.
Python + NumPy. See: python-code.d/neural/TestMultilLayer-XOR-Gate01.py
TensorFlow 2 + Keras. See: python-code.d/tensorflow/TestKeras-XOR-Problem.py
Deeplearning4J. See: src/nn/basic/BooleanGateXor.java
Example 4: Modeling XNOR and XOR Boolean Gates.
TensorFlow 2 + Keras. See: python-code.d/tensorflow/TestKeras-XNOR-XOR-Problem.py
Deeplearning4J. See: src/nn/basic/BooleanGateXnorXor.java
Example 5: Points in a Convex Polygon.
Deeplearning4J. See: src/nn/basic/BooleanConvexPolygon.java
Counter Example 6: Points in a Non-Convex Polygon.
Deeplearning4J. See: src/nn/basic/BooleanNonConvexPolygon.java
Example 7: Points in a U-Shaped Polygon.
Deeplearning4J. See: src/nn/basic/BooleanNonConvexPolygon.java
Example 8: Neural Network for Digit Recognition.
Work in progress: ....
A zip file of DeepLearning4J (DL4J) code and examples will be distributed this week.
Check your e-mail for a message from Dropbox.
Project Abstracts
I want to create a web page for the project abstracts, which follows the format used in Fall Semester 2020 .
Please send me:
- The title of your project.
- The authors of the work (up to 2 students).
- A brief description -- 5 to 10 sentences -- of the projects goals.
- A picture to go with your abstract.
- The sources of data and a handful of references.
We welcome projects that can continue beyond this class and lead to conference and/or journal publications.

Week 13: April 18 and 20, 2022.

Topic (2022-04-18): Project Pitch Presentations
Presentation Guidelines:
Duration: Aim for 15 minutes; No more than 20 minutes.
Slides: No more than ten slides.

Content: Project Title.
What are we are doing? Why is the problem is important?
Here's how the project relates to data mining and machine learning?
What are the data and computational challenges?
What will project success look like?
Presentation Schedule
- Monday (18th): Yi-Ting and Yen-Lin Huang.
- Wednesday (20th): Adriana and Caitlin.

Week 14: April 25 and 27, 2022.

Presentation Schedule
- Monday (25th): Hanson Liao, Ainur Abilbayeva and Naiyi Li, Ahmet Darilmaz, Fei Wu, Timothy Henderson.
- Wednesday (27th): Jing Jing.
Handout: TensorFlow: A System for Large-Scale Machine Learning (pdf),
Proceedings of 12th USENIX Symposium on Operating System Design and Implementation (OSDI 2016),
Savannah, November, 2016.
Follow-up to our discussions during office hours:
Reading. Lagaris I.E., Likas A., and Fotiadis D.I., Artificial Neural Networks for Solving Ordinary and Partial Differential Equations (pdf),
May 1997.
Reading. Chen T.Q.C., Rubanova Y., Bettencourt J., and Duvenaud D., Neural Ordinary Differential Equations (pdf),
32nd Conference on Neural Information Processing Systems (NIPS), Montreal, Canada, 2018.
Reading. Lample G., and Charton F., Deep Learning for Symbolic Mathematics (pdf),
Facebook AI Research, arXiv:1912.01412v1, December 2019.
Physics-Informed Machine Learning. Two observations:
- Standard approaches to engineering analysis make heavy use of physics and little use of data.
- State-of-the-art neural networks are all about learning from big data and abstract away explicit consideration of physics.
Since neither of these approaches is ideal, an emerging and important area is development of computational methods that strike a middle gound:

Presentation Slides. Perdikaris P., Physics-Informed Deep Learning (pdf),
University of Mennsylvania, April, 2020.
Solving Differential Equations with Neural Networks.

See this interesting link on Github.
DeepXDE: Deep learning library for solving differential equations.

Weeks 15-16: May 2, 4 and 9, 2022.

Tutorials in Progress (Spring 2022 ...):
Machine Learning Software and Tools: Python/Java Software Setup, Tools, etc (pdf) (draft) (Version: 2022-06-05).
Python Tutorial -- Part I: Introduction (pdf) (draft) (Version: 2023-01-18).
Python Tutorial -- Part 2: Objects and Classes (pdf) (draft) (Version: 2023-01-13).
Data Science: Techniques and Tools (pdf) (draft) (Version: 2022-11-21).
AutoEncoders I (pdf) (draft) (Version: 2021-12-01).
AutoEncoders II (pdf) (draft) (Version: 2021-11-27).
Recurrent Neural Networks (pdf) (draft) (Version: 2022-12-07).
Machine Learning Appendices: Tensors etc (pdf) (draft) (Version: 2022-06-05).
Final Project Presentations
Presentation Guidelines:
Duration: Aim for 15 minutes; No more than 20 minutes.
Slides: No more than ten slides.

Content: Project Title.
What are we are doing? Why is the problem is important?
Here's how the project relates to data mining and machine learning?
Describe the data and computational challenges?
What will long-term project success look like?
Project Presentation Schedule
- Wednesday (May 4): Hanson Liao, Timothy Henderson, Yi-Ting and Yen-Lin Huang.
- Monday (May 9): Adriana and Caitlin, Jing Jing, and Ainur Abilbayeva and Naiyi Li, Fei Wu, Ahmet Darilmaz.
Term Projects
Term projects will be due Friday, May 13.
Please e-mail me a pdf of your project report.
I would aim for around 25-35 pages. No extensions!
Report Contents: What problem is the project trying to solve? Why is the problem is important?
Describe the data and challenges in working with the data?
What role does data mining and/or machine learning play?
Preliminary results, conclusions and future work