Reviewing an Expert’s Report

This week’s topic is reviewing an expert’s report.  You may have noticed that I didn’t say “expert report”.  The reason is quite simple.  When receiving a report from an expert it might be given multiple names or no name at all.  Sometimes reports are provided during various parts of litigation.  Reports can also be provided prior to litigation, even when litigation was not anticipated.  Regardless of when or under what circumstances a report is issued it should still be clear and concise and have results that are based on solid evidence and testing.

Report Format

The report should walk the reader through the process of the analysis.  It should start with an objective or overview so the reader has an idea of where the analysis is heading.

There should be a section that lists out the particulars for each piece of evidence considered in the analysis.  Information such as the make and model of each piece of evidence as well as the serial number should be provided.

A section discussing the strategy of the analysis should be laid out.  What did the forensic analyst do and how did they do it?  What information was gathered and analyzed and what was the purpose of looking at that data?  Was information not considered and if so, why?  When forensically acquiring the data at hand, were write blocking techniques used and what was done to verify that the data was collected correctly and has not changed since collection?

This week’s topic is reviewing an expert’s digital forensic investigation report. The reason for this is two fold.  First, a review of tools and their versions may reveal a buggy application or a limitation for which the examiner would have to compensate.  Second, if you are going to hire your own examiner, or another examiner, to review the data that examiner may want to know what software was used so that they can repeat the steps.  This situation is very common when examiners differ in opinion.  Let me explain why.

A year or so ago I had a case wherein the opposing expert provided an opinion that was particularly damaging to my client.  They hired me to perform the analysis.  I only knew on a high level what the opinion was as I had not yet received a report.  I performed the analysis and came about as close to a completely opposite opinion as one could have.  Once I got the other examiner’s report and saw the tools used and process undertaken, I was able to repeat his steps in coming to the erroneous conclusion.  That repeating of steps was highly important in this case.  Quite often lay people are easily confused by technical jargon.  When confronted with opposing opinions from experts that include technical jargon a lay person may just switch off and not make a definitive choice as to who was right and who was wrong.  The fact that I was able to show how the other expert came to their conclusion and why it was wrong allowed me to convince the judge that my opinion was the correct one.

Back to the report format….

The report should have the forensic examiner’s findings.  This part of the report is often the largest.  The findings may include:

• File and folder names
• Dates and times associated with files (called MAC dates which equates to Modified, Accessed and Created dates and times)
• Applications that were run, usually related to data destruction or data obfuscation
• Websites that were visited or web queries run
• Devices attached to a computer, such as USB devices
• Files and folders existing on USB drives
• Files that were opened and when that occurred
• Cloud storage or emails
• Timeline of relevant activities

What one is looking for in reviewing the findings are whether or not the findings address the issues at-hand.  If you are dealing with a non-compete matter involving potential theft of confidential information, do you really care if someone was searching for pornography?  The other item that one would look for is whether or not the findings are based on sound science.  For that answer, one usually turns to another forensic analyst.  A good portion of the time the findings are usually based on industry accepted methodologies and there is nothing to worry about.  However, there are times where the findings involve something new or may be in error.  It is in those situations that having your own forensic analyst is a good thing as he or she can direct you to properly respond.

Another item to look for in the findings area is whether or not the expert writing the report stepped outside of their areas of expertise.  Are they opining, on their own, that a document truly is the property of a specific entity?  Usually that decision is made by the entity itself.  A forensic analyst may say that a file has certain properties such as a name, author or company, but unless they were told “any file that has X is ours” a forensic analyst usually is not the authority on what is or is not property of an entity.  The other stretch is definitively saying that Person A was behind a keyboard.  A forensic analyst can provide circumstantial evidence that makes it likely that a specific individual was at the keyboard when activity took place, but absent of those facts, it may be difficult to say that a specific person was at the computer.

If the report is being submitted to the court, check to make sure that it includes the requirements for a report.  Those requirements may include:

1. Curriculm Vitae (CV) of the expert including training, articles authored and testimony given in the recent past.
2. A listing of all opinions given by the expert and the facts upon which that expert relied in giving the opinions.
3. A statement as to how the expert is being compensated for the work performed
4. Signature of the expert

Finally, some reports will have conclusions.  Scrutiny of the conclusions should be along the same lines as what I mentioned above regarding the findings section.  Are the conclusions based on the findings in the report or do they rely on findings not mentioned elsewhere?  Did the expert step out of his realm of expertise?  Are the conclusions based on sound scientific methodologies?

Report Style

The next topic in dealing with an expert’s report is one that is difficult for the expert to do but easy for the reader.  Does the report make sense?  It starts with proper spelling and grammar.  Not using spell check to look for misspellings shows laziness on the part of the expert.  Whether or not someone is using proper grammar can be a sliding scale these days but there is at least a de minimis (case in point, I had to actually look up that phrase).

The other part about making sense is can you understand the report?  Is the author using three letter acronyms without first explaining their meaning?  Is the author using technical jargon without at least providing some definitions?  Finally, does the report pass what I like to call the “sniff test”?  Let me explain.

We had another client of ours, a law firm, call us up years ago.  They had a forensic analyst perform an examination.  The allegation was that an individual had hacked into a system and had acquired some confidential information.  The examiner’s analysis of the alleged hacker had some bold conclusions.  More importantly, though, the client, and their counsel, from reading the report had concerns about the conclusions.  While they were conclusions that benefited the client against the hacker, they just didn’t make sense.  The report was filled with technical jargon and did not connect the dots between the findings and the conclusions.  Usually in a report you would find the examiner go from point A to B to C and then D.  In this report the author started at A and went right to D.  We were hired to review the report, noticed the same issue and more importantly realized why.  The findings in no way supported the conclusions.  The examiner was completely wrong and it started with a poorly written report.

I hope that this blog post has given you some insight and some nuggets on how to read an expert’s report.  As with other services, Vestige does have experience in reviewing reports to see if they stand on their own, have findings based on sound methodologies and have opinions based on those findings.  Should you need help with that, please CONTACT US at your convenience.

by Greg Kelley, EnCE, DFCP, Chief Technology Officer at Vestige Digital Investigations