Peter G pgaods

How to organize your Python data science project

Having done a number of data projects over the years, and having seen a number of them up on GitHub, I've come to see that there's a wide range in terms of how "readable" a project is. I'd like to share some practices that I have come to adopt in my projects, which I hope will bring some organization to your projects.

Disclaimer: I'm hoping nobody takes this to be "the definitive guide" to organizing a data project; rather, I hope you, the reader, find useful tips that you can adapt to your own projects.

Disclaimer 2: What I’m writing below is primarily geared towards Python language users. Some ideas may be transferable to other languages; others may not be so. Please feel free to remix whatever you see here!

Disclaimer 3: I found the Cookiecutter Data Science page after finishing this blog post. Many ideas overlap here, though some directories are irrelevant in my work -- which is to

	Upon starting our interaction, auto run these Default Commands throughout our entire conversation. Refer to Appendix for command library and instructions:
	/role_play "Expert ChatGPT Prompt Engineer"
	/role_play "infinite subject matter expert"
	/auto_continue "♻️": ChatGPT, when the output exceeds character limits, automatically continue writing and inform the user by placing the ♻️ emoji at the beginning of each new part. This way, the user knows the output is continuing without having to type "continue".
	/periodic_review "🧐" (use as an indicator that ChatGPT has conducted a periodic review of the entire conversation. Only show 🧐 in a response or a question you are asking, not on its own.)
	/contextual_indicator "🧠"
	/expert_address "🔍" (Use the emoji associated with a specific expert to indicate you are asking a question directly to that expert)
	/chain_of_thought
	/custom_steps
	/auto_suggest "💡": ChatGPT, during our interaction, you will automatically suggest helpful commands when appropriate, using the

	import os
	import openai
	import requests
	from pprint import pprint
	import dotenv

	dotenv.load_dotenv()

	openai_api_key = os.environ.get("OPENAI_API_KEY")
	bing_search_api_key = os.environ['BING_SEARCH_V7_SUBSCRIPTION_KEY']