![Page 1: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/1.jpg)
![Page 2: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/2.jpg)
Open materials & methods
• Increases the ability to run replications
![Page 3: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/3.jpg)
Open data and code
• Increases the ability to check for computational reproducibility
• Increases the ability to determine the robustness of the statistical inference
![Page 4: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/4.jpg)
Open research workflow
![Page 5: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/5.jpg)
What is an open workflow?
• Research inputs and outputs are open:– Data, Code, Materials, Protocols, etc.
• The evolutions of these inputs and outputs over the research process– How and when versions of files changed
![Page 6: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/6.jpg)
Open research workflow
• Increases ability to run replications
• Increases the ability to check computational & statistical reproducibility
• Increases the ability to detect researcher degrees of freedom
![Page 7: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/7.jpg)
What is required?
• Increase open sharing
• Increase documentation
• Incentives
![Page 8: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/8.jpg)
![Page 9: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/9.jpg)
Enabling Open Workflows
Open Science Frameworkhttps://osf.io
![Page 10: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/10.jpg)
Project Overview Page
![Page 11: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/11.jpg)
Versioning
![Page 12: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/12.jpg)
Registering Your Work
![Page 13: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/13.jpg)
Sharing Your Work
![Page 14: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/14.jpg)
Incentivizing Open Practices
• Badges for open practices
![Page 15: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/15.jpg)
Badges to Acknowledge Open Practices
![Page 16: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/16.jpg)
Badges to Acknowledge Open Practices
![Page 17: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/17.jpg)
Incentivizing Open Practices
• Badges for open practices
• Additional metrics
![Page 18: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/18.jpg)
Accessing Statistics
![Page 19: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/19.jpg)
Search and
discover
Develop idea
Design study
Acquire materials
Collect dataStore data
Analyze data
Interpret findings
Write report
Publish report
![Page 21: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/21.jpg)
Reproducibility Projects
![Page 22: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/22.jpg)
Many Labs Project
![Page 23: Open materials & methods · 2017. 1. 19. · Retro. gambler's fallacy (Oppenheimer & Monin 2009 Gain vs loss framing (Tversky & Kahneman 1981 Sex diff. in implicit math attitudes](https://reader033.vdocument.in/reader033/viewer/2022060715/607b28f67833c926436ff15a/html5/thumbnails/23.jpg)
Crowdsourcing Science