University of Kansas Electrical Engineering & Computer Science

EECS 861 - Random Signals and Noise

Announcements:

13-Homework Due Dec 10, 2024 Only problem 1 due 12/10. Problems 2 & 3 are optional and due 12/12 however they will not be graded and will not be returned. However, solution to Problems 2 & 3 will be provided after class on 12/12. The materiel covered by Problems 2 & 3 is part of the topics to be covered on the final.

EECS 861 Final December 16 10:30 - 1:00

 

________________________________________________________________________________________________________________________

Test 2

In class review for Test 2 on Tuesday, Dec 3, 2024

Open review Monday, Dec 2, 5:00-6:50 PM in LEA 1136 - Learned

Test 2 Thursday, Dec 5, 2024

Open Book/Open Notes

No laptops, tables, or phones

Only paper notes/book

No points will be given on entire problem if answer given is P>1 or P<0 or var<0 without acknowledgment of calculation error

Covers homework assignments 8-11

Topics for Test 2

 

________________________________________________________________________________________________________________________

Test 1

• Open review Monday, Oct 21, 5:00-6:50 PM in 1136 Learned

• In class review for Test 1 on Tuesday Oct 22, 2024

• Test 1 Thursday Oct 24, 2024

• Open Book/Open Notes

No laptops, tables, or phones

Only paper notes/book

• Covers homework assignments 1-7

• Topics for Test 1

___________________________________________________________

Recommended reading: The Signal and the Noise: Why So Many Predictions Fail--but Some Don't by Nate Silver

________________________________________________________________________________________________________________________

Handouts

Multivariate Gaussian

MAP, MS, and ML estimators for A where Z=A+N and A & N are Gaussian

Harmonic Addition Theorem

Decorrelating and then Whitening data

Eigenvalues and Eigenvectors

Poisson Process and the Exponential pdf

Binary Detection: Multiple Independent Observations

Detection, decisions, hypothesis testing

 

• Course Syllabus
• Homework Format
• Discussion 5:00 to 6:50 Mondays in 1136 Lea will be used for test reviews, make up classes, and as needed homework reviews. Will not meet every week; check web if discussion session is meeting.

Homework

Interactive Graphs (using Wolfram cdf format: Download Wolfram CDF Player)

This course will not utilize +/- grading in Fall 2023.

Plotting Links

• Plotting Functions in MatLab using fplot
• Creating stem plots in MatLab

Introduction Links

• Signal plus Noise
• Birth/Death Process-Statistical Multiplexing
• Circuit Switch Model
• Audio Spectrogram

Probability Links

• Playing Cards
• Dice- Equally Likely
• Dice-Relative Frequency
• Multiple Multiply Sided Dice Probability
• Binomial Probability Distribution
• np approximation for 1 or more error in n bits
• Example of Joint pmf
• Probability of Being Sick After Having Tested Positive for a Disease (Bayes's Rule)
• Correlation and Covariance of Random Discrete Signals
• Individual and Cumulative Binomial Probabilities
• Poisson Distribution
• Exponential Distribution
• Continuous Uniform Distribution
• Gamma Distribution
• Log Normal Distribution
• Examples of Characteristic Functions
• Percentiles of Continuous Probability Distributions with CDFs and pdfs
• Gaussian Probability Calculator
• Chebyshev Bound for the Double Exponential Distribution Function
• Chernoff Bound with Example
• Probability Distributions (with Pareto) of Some Relationships Between Random Variables

Gaussian Distribution Links

• Q function table
• Normal Distribution
• Finding Probabilities for Intervals of a Normal Distribution
• Connecting the CDF and the PDF
• On-line Normal Distribution Calculator
• Inverse Q function
• Joint Density of Bivariate Gaussian Random Variables
• Scatter Plot for 1 Dimensional Gaussian RVs
• Bivariate Gaussian Scatter Plot
• The Bivariate Normal and Conditional Distributions
• Conditional Probability P(X>L|Y=y) with X & Y Bivariate Gaussian Random Variables
• Scatter Plot for 3 Dimensional Gaussian Random Vector
• Conditional Gaussian distributions from a 2-Dimensional Mulitvariate (Bivariate) Gaussian
• Conditional Bivariate distributions from a 4-Dimensional Mulitvariate Gaussian
• Conditional Bivariate distributions from a 3-Dimensional Mulitvariate Gaussian
• Dynamic Scatter Plot for 2 & 3 Dimensional Gaussian Random Vectors
• Example of Transformation of Bivariate Gaussian Random Vector
• Diagonalising Covariance Matrix-Class Example
• Diagonalizing Covariance Martix and Whiten-Class Example
• Whitening Bivariate and 3-dimensional Gaussian Random Vectors General Cases
  • Whitening Bivariate Gaussian Random Vector
  • Whitening 3-dimensional Gaussian Random Vector-Fixed Covariance Martix
  • Whitening 3-dimensional Gaussian Random Vector- Covariance Martix from MA-2 Process
• Central Limit Theorem for the Continuous Uniform Distribution
• Chi-Squared Distribution and the Central Limit Theorem
• Illustrating the Central Limit Theorem with Sums of Bernoulli Random Variables

Random Processes Links

• Example Random Process
• Random Processes with histograms at time t
• Speech Example with Quantizing
• Random Cosine Process
• Random Binary Waveform
• Chapter 3 Example 7
• Two-State, Discrete-Time Markov Chain-Member Functions
• Two-State, Discrete-Time Markov Chain-Member Functions with E[X[n]X[n+k]]
• A Two-State, Discrete-Time Markov Chain
• Finite-State, Discrete-Time Markov Chains
• Interactive Gaussian Process Visualization
• Study of the Conditional probability P(X(t+Tau)>L|X(t)=y) for Gaussian Random Process given Different Autocorrelation Functions - with variance
• Study of the Conditional probability P(X(t+Tau)>L|X(t)=y) for Gaussian Random Process given Different Autocorrelation Functions - with joint pdf
• Comparison of Interactive Visualization and Study of the Conditional probability P(X(t+Tau)>L|X(t)=y) with joint pdf
• Random Telegraph Process
• Time Series Viewer
• Power Spectral Density of Random Sequence
• Frequency Spectrum of a Noisy Signal
• Poisson Process
• Random Walk Process
• Low Pass and Bandpass Gaussian Random Processes
• Wiener Process
• Solving for the variance of a time average
• Example 3.19 from Book
• Integrate and Dump with BER Calculation
• Karhunen–Loève (KL) expansion
• Sampling a Random Process
• Quantizer
• PSD of Moving Average with Equal Weights
• Second Order Moving Average Process with Block Observations
• Second Order Moving Average Process with Sliding Window Observations
• Autoregressive Moving-Average Simulation (First Order)
• Auto-Regressive Simulation (Second-Order)
• ARMA Study-MA(1), MA(2), AR(1), ARMA(1,1)
• ARMA Modeling of Speech
• Autoregressive Moving-Average Generator
• Power Spectral Density of Narrowband Noise

Filtering, Detection and Estimation Links

• Binary Detection with Gaussian Noise
• Speech Processing
• Critical Value z* for z-Scores for Confidence Levels
• Filtering a White-Noise Sequence
• Receiver Operating Characterstic (ROC) for Gaussian Noise
• MAP detection with Signal and Noise Exponentially Distributed
• Signal Detection Theory (Hypothesis Test)
• Concept of 4-ary Detection-One Dimensional
• Concept of 4-ary Detection-Two Dimensional
• Concept of 8-ary Detection-Three Dimensional
• Symbol Detection (3 samples/symbol) in White Noise Calculation of Symbol Error Rate
• General Symbol Detection in Correlated Noise-2 samples/symbol Calculation of Symbol Error Rate
• General Symbol Detection in Correlated Noise-3 samples/symbol Calculation of Symbol Error Rate
• General Symbol Detection in Correlated Noise-8 samples/symbol Simulation and Calculation of Symbol Error Rate
• General Symbol Detection in Correlated Noise-8 samples/symbol Analytical Calculation of Symbol Error Rate
• Performance of Continous Time Matched Filter
• MAP Estimator with Gaussian Prior and Gaussian Noise
• Confidence Intervals for a Mean
• Wiener Filter for discrete MA(2) Signal in White Noise
• Approximate Wiener Filter for discrete AR(1) Signal in White Noise
• Example 7.14 from Shanmugan and Breipolh Digitial Wiener Filter with Signal MA(2)
• Kalman Filter for AR(1) Signal

Reference Information

• Topic-Decorrelating and then Whitening data
• Hand-book on STATISTICAL DISTRIBUTIONS for experimentalists

---

Academic Integrity and Plagiarism The department, school and university have very strict guidelines regarding academic misconduct. Obviously, copying is not allowed on exams. Students are expected to submit their own work on individual homework and projects. Lending or borrowing all or part of a simulation model or program from another student is not allowed. Students ARE allowed to borrow and modify any code on this class web site in their projects. Instances of cheating will result in a referral to the department chairman and the dean of engineering.
All sources in your written work (project reports) must be properly referenced; if you use a source from the literature or the idea of another for your work you must reference it. If you quote or copy a block of text, it must be cited and included in quotation marks (if a sentence or less in length) or in block quote style (if more than a sentence in length). If you paraphrase text (reword a phrase, sentence, or paragraph), you must also quote or blockquote followed by “[paraphrased]” in addition to proper citation. Figures taken from other sources must be referenced.

The USC academic integrity quiz is also useful reading. If you have any doubt, talk to me – inexperience in past writing or coming from an environment where plagiarism was permitted will not be an acceptable excuse for academic misconduct.

I recommend that you take intermediate notes from which you write your own words. I strongly recommend that you not write in one window while displaying the work of others in another window; this is asking for trouble. “Unintentional” paraphrasing is also not an acceptable excuse for academic misconduct.

Modified with premission from James P.G. Sterbenz and John Gauch

• Use of EdTech services

  • Professors and instructors at the KU School of Engineering are aware that some students are actively posting homework, laboratory, and exam questions and responses to EdTech services (e.g., Chegg) even during exam time frames.
    Keep in mind that when a person signs up to participate by either uploading, and/or downloading, and/or using posted material from these sites, the “terms of service” that are agreed to do not protect the person when KU and/or the School of Engineering decide to conduct investigations related to academic misconduct (e.g., plagiarism and/or cheating).
    In fact, EdTech services, like Chegg, retain contact information of students who use their services and will release that information, which is traceable, upon request. Using these services constitutes academic misconduct, which is not tolerated in the School of Engineering. It violates Article 3r, Section 6 of its Rules & Regulations, and may lead to grades of F in compromised course(s), transcript citations of academic misconduct, and expulsion from the University of Kansas.
    If unsure about assignments, it is important that students use the allowable available resources, such as instructor office hours, graduate teaching assistants, and/or tutoring. The School of Engineering wants students to be successful; cheating is not the way to attain that success.
• Use of smartphones, tablets, and laptops in class
  • Smartphones, tablets, and laptops may only be used in direct support of class activities
  • Texting, general web browsing, checking of e-mail is NOT permitted during class
  • Video and audio recording of the EECS 861 class lectures is prohibited.
• Changes announced in class and the class web site supersede these written instructions.
• Advice on Writing
• Most colleges and universities have a writing center, a place for students to talk about their writing with trained peer consultants. At KU, we call our writing centers Writer's Roosts. When you visit, bring your work in progress and an idea of what you would like to work on-organization, support, documentation, editing, etc. The Roosts are open in several different locations across campus; please check the website at www.writing.ku.edu for current locations and hours. The Roosts welcome both drop-ins and appointments, and there is no charge for their services. For more information, please call 864-2399 or send an e-mail to writing@ku.edu

Author

Victor S. Frost, vsfrost@ku.edu