Computer Science Retention Study

Data Collection UtilityEric L. Brown

Purpose

• Develop a tool to get raw grade data from TTUMIS for local processing.

• Preprocess the raw text files into a SQL Server 2005 RDBMS.

• Develop a framework that can be easily extended to include other data sources as required by the study.

Use as many open source tools as possible.

Parts of the Puzzle (Part 1 - The Process)

• Raw Data Collection

• Establishing Performance Metrics

• Model Data via Established Metrics

• Dissemination

Maintenance

Parts of the Puzzle(Part 2 - the helpers)• pexpect-2.1beta (pexpect.sf.net)

• freetds-0.63 (freetds.org)

• pymssql-0.7.4 (pymssql.sf.net)

• PyXML-0.8.4 (pyxml.sf.net)

• Gnosis_Utils-1.2.1 (gnosis.cx)

FCKEditor-2.2 (fckeditor.net)

Parts of the Puzzle(Part 3 - da systems)

Parts of the Puzzle(Part 4 - the scripts)

• createtables.py / parse.py / samplereport.py

• index.py / form_action.py

• mathdata.py / getdata.py

dataeditor.py / dataeditorpost.py

Security

• Security before all else...

• Apache 1.3.33 / SSL

• ssh / sftp

• will have encrypted connection to MS-SQL

not on a public system

Demo

Conclusions

• I love Python for a scripting language, but be aware of which version you are running.

• There is more security work to be done.

• Only small requests can be run through the browser window due to timeouts.

Web will be mostly for reporting in future releases.

Thank You!

• Questions

• Questions

No Question