Data Analysis with Python | Nesqual TechCourse outline
The analysis workflow and the Fernhill data
- What data analysis is, and what it is notLesson25 min
- The toolkit: the standard library and the tools used at workLesson25 min
- Meeting Fernhill Bikes and its dataLesson30 min
- Data shapes: records, columns and tidy dataLesson30 min
- Asking good questions and defining metricsLesson30 min
- Reproducible analysis habitsLesson30 min
- Chapter quiz: the analysis workflow and the Fernhill dataQuiz15 min
Loading, parsing and typing data
Reading a file for analysis: look before you trustLesson30 minConverting types: from text to numbers you can trustLesson30 minParsing dates and times without guessingLesson30 minJSON and nested data: from trees to tablesLesson30 minA typed loader: one schema, one function, one error reportLesson35 minAwkward sources: semicolons, decimal commas, fixed width, gzip and ExcelLesson30 minChapter quiz: loading, parsing and typing dataQuiz15 minCleaning and data quality
- Data quality dimensions and profilingLesson30 min
- Duplicates: exact, key and nearLesson30 min
- Inconsistent categories and textLesson30 min
- Missing values: drop, fill or flagLesson30 min
- Invalid values and domain rulesLesson30 min
- A cleaning pipeline with an audit trailLesson35 min
- Quiz: cleaning and data qualityQuiz15 min
- Lab: profile and clean the Fernhill tripsLab40 min
Describing data with statistics
- Centre: mean, median and modeLesson30 min
- Spread: range, variance, standard deviation and IQRLesson30 min
- The five-number summary and your own describe()Lesson30 min
- Proportions, rates and denominatorsLesson30 min
- Comparing groups descriptivelyLesson30 min
- Outliers as statistics: z-scores, fences and trimmed meansLesson30 min
- Describing data quizQuiz15 min
Distributions and charts in the terminal
- Histograms: seeing how values are spreadLesson30 min
- Bar charts for categories and the hbar helperLesson30 min
- The shape of a distribution: skew, peaks, logs and box plotsLesson35 min
- Sequences: sparklines and line chartsLesson30 min
- Scatter plots in text: two variables at onceLesson30 min
- Charts at work and charting honestlyLesson30 min
- Quiz: distributions and text chartsQuiz15 min
- Lab: three charts for the operations managerLab40 min
Grouping, pivots and combining datasets
- Derived variables: weekday, part of day, bands and districtLesson30 min
- Grouping by several keysLesson30 min
- Several aggregates per groupLesson30 min
- Pivot tables, cross-tabs and reshapingLesson35 min
- Joining datasetsLesson35 min
- sqlite3 as an in-memory analysis engineLesson30 min
- Quiz: grouping, pivots and joinsQuiz15 min
- Lab: the district table, the daily weather join and station ratesLab40 min
Working with time
- Time as data, time zones and the calendarLesson30 min
- Resampling: trips per day, week and monthLesson30 min
- Patterns within the week and the dayLesson30 min
- Rolling windows and cumulative totalsLesson30 min
- Measuring change over timeLesson30 min
- Pitfalls of time dataLesson30 min
- Quiz: working with timeQuiz15 min
- Lab: weekly rates, a weekday heatmap and rolling averagesLab40 min
Relationships between variables
- Two variables at onceLesson30 min
- Pearson correlationLesson30 min
- Rank correlationLesson30 min
- A straight line as a descriptionLesson35 min
- Correlation is not causationLesson35 min
- Relationships between categoriesLesson35 min
- Quiz: relationships between variablesQuiz15 min
- Lab: weather and demand at FernhillLab40 min
Sampling, uncertainty and comparing groups
- Samples and populationsLesson30 min
- How much an estimate wobblesLesson30 min
- Confidence intervals and the bootstrapLesson35 min
- Hypothesis testing with permutation testsLesson35 min
- Effect size, practical significance and an A/B trialLesson35 min
- Honest uncertaintyLesson30 min
- Chapter quiz: sampling, uncertainty and comparing groupsQuiz15 min
Larger data and performance
- When lists of dicts get heavyLesson30 min
- Streaming and one-pass aggregationLesson30 min
- A bigger dataset, compressed and partitionedLesson30 min
- Measuring speed honestlyLesson30 min
- Pushing work into SQLiteLesson30 min
- When to move to other toolsLesson30 min
- Larger data and performance quizQuiz15 min
Reporting and communicating findings
- From analysis to messageLesson30 min
- Tables people can readLesson30 min
- A reproducible report scriptLesson35 min
- Charts in reportsLesson30 min
- Honesty, privacy and ethicsLesson35 min
- Notebooks and reviewLesson30 min
- Reporting and communicating findings quizQuiz15 min
Capstone: the Fernhill spring review
- The brief: five questions from the operations managerLesson25 min
- Planning the review and building the shared tablesLesson30 min
- A worked question: station capacity pressureLesson30 min
- Common capstone mistakes and how to catch them yourselfLesson30 min
- The capstone translated: pandas and matplotlib beside the standard libraryLesson30 min
- Where next: from describing demand to predicting, querying and automating itLesson25 min
- Capstone quiz: judging a reviewQuiz15 min
- Lab: the Fernhill spring reviewLab40 min