Project Evaluation

A strong technical report

Addresses a meaningful and interesting question, which (if solved) could produce some action in the world. It contains technically correct interpretation of regression coefficients, as well as discussion about the practical significance of those results. Strong reports contain appropriate statistical techniques (cleaning data, checking conditions, looking for multicolinearity, performing ANOVA on complex models) but describe the results in plain language. They contain the most important elements of descriptive data analysis, without running through a laundry list of every variable and its corresponding statistics.

A strong report will address all the regression conditions, either showing with pictures and written description they are met, or explaining why they are not. The strongest reports address issues in adherence to regression conditions by performing transformations of the data, and addressing the corresponding change in the coefficient interpretation.

Strong reports contain an abstract that addresses both the problem and the biggest takeaways from the model, an introduction going into more detail about the context of the data, and a conclusion bringing together model results and context to provide action items for the stakeholders who would care about the analysis. A strong conclusion describes future directions for work, but does not use this as a crutch to avoid implementing those elements.

Strong reports contain outside references, both citing where the data came from, and providing additional context for the problem at hand.

An adequate technical report

Addresses an uninteresting question, which (if solved) would not produce any meaningful action in the world. Contains technically correct interpretation of regression coefficients, but may not discuss practical significance. Contains some appropriate statistical techniques, but does not explain terminology to demonstrate deep understanding of why those techniques were used. Adequate technical reports often contain descriptive statistics of variables, but statistics are not folded into the narrative of the report, simply listed without context.

Adequate technical reports check regression conditions, but if they are not met, just move on.

An adequate technical report often relies heavily on describing potential future directions for work, rather than actually implementing the ideas, such as data transformations or finding additional variables. Conclusions are often divorced from the context of the data, or do not lead to meaningful action on the part of the stakeholders.

A poor technical report

Attempts to answer a question, but fails to address major statistical issues (e.g. multicolinearity, lack of independence, non-random sampling, etc) and therefore cannot draw any conclusions. If conclusions are drawn, they are faulty, and coefficient interpretation may be technically incorrect. Often, poor technical reports attempt to draw inference to an inappropriate population.