Working at APPRISE: My Transformation, Experience, and Advice
When I started working with APPRISE, I had just finished my master’s degree in behavioral and computational economics and had a very technical, academic background. I had very recently decided not to do a PhD and most of my coursework focused on very technical programming, mathematics, and statistics courses. I wanted to write this piece to outline what I’ve learned, what my work experience has been, and any advice I have for someone starting their career at APPRISE.
What I've Learned
Skepticism – When I started, I was hyper-focused on why evidence was imperfect and therefore not useful. At APPRISE, I learned skepticism is fine, but I also need to acknowledge what you CAN learn from evidence instead of only seeing its drawbacks. I now question what I would expect to change, or even what could change, from implementing a fancier, more complicated method. Most of the time, the answer is “not much”, and I feel it’s more responsible to implement the simpler method.
Writing Style – When I arrived at APPRISE, I wrote with a very informal tone – much like this post – but with tons of jargon. While I think it’s fine to write in an informal tone, there’s a time and a place for everything, and a research report is not the right time. Additionally, using jargon quickly communicates an idea to a small subset of people. It’s not very helpful to write something like, “To address endogeneity concerns with our model, we…,” when you could just as easily write, “Because we’re unsure if A causes B or B causes A, we…” Did the second option capture all the nuances of endogeneity? No. Did it communicate what I needed to communicate? Yes, and it does not require someone to have taken an econometrics course to understand the sentence.
Writing Emails – If someone asked me what skill I am most happy to have improved during my time at APPRISE, I would not respond with my understanding of survey methodology and degree-day normalization or my ability to keep track of multiple demanding projects simultaneously – though I am very happy to have improved my skills in these areas. My most cherished skill is how to write a good email. I used to write 1000-word essays in emails to explain every nuance of my thought process and all the pros and cons of each decision which needed to be made. However, people only want to spend 20 seconds on your email, not 20 minutes. Writing a good email means ALL the information you need to get across is there, but the reader can identify the relevant information and make an informed decision in less than one minute.
Documentation and Clean Code – Over the last year, I have worked on well over a dozen projects for APPRISE. However, I don’t work on any of these projects alone. Additionally, once I finish a task, there’s no guarantee I’ll be the analyst working on the next stage of the project. Initially, it was very common for one of my colleagues to come to my desk and ask me questions about previous work. I thought this meant I was important because I had the answers. I have since learned this meant I wasn’t doing my job very well. When you work in a collaborative environment, it is essential to document your work and write clean, easily understandable programs. This is not a luxury; it is a necessity.
Mistakes – I am a fervent believer that mistakes are not the end of the world and reflecting on mistakes can lead to a lot of growth. However, as written above, there is a time and a place for everything. When you are working through the first draft of a novel program, make all the mistakes you want! Trial and error can be a very useful method to improve your skills, explore, and deepen your understanding of your data. However, in our line of work, sending out a report with mistakes is unacceptable. If we make mistakes, there are real consequences. Budgets can be misallocated, analysts working at APPRISE or elsewhere could become horribly confused, or programs can be misguided for years to come. At APPRISE, I’ve learned, “It’s always worth it to spend 5-15 minutes to check for mistakes.”
Stata – Programming in Stata required a different type of thinking than programming in Python (my first programming language). One of my catch phrases during my first few months at APPRISE was, “This is really easy to do in Python, but I have no idea how to do this in Stata.” Once I started reviewing other analysts’ programs, I learned just how flexible Stata is as a language and now I find it easier to manipulate data in Stata than in Python or R. A bit of advice – don’t fight the programming language you’re working with; learn it and play to its strengths.
“Working Memory” – This what I call the ability to remember seemingly useless information which tends to serve as critical knowledge later point. There have been times where I’m in a meeting with Jackie and she asks me if I know one of the 150 formulas I’m working with off the top of my head, and somehow, I do! There have been times where David catches me walking back from lunch, needs an answer to a miniscule detail so he can continue with his work, and I’m somehow able to answer it without reviewing my notes. Trust me, I’m not that smart and this was NOT a skill I had when I arrived at APPRISE. But somehow the level of detail we maintain and the thoroughness with which we prepare for everything we do has translated into developing my “working memory” and it feels like a superpower!
Would I have been able to learn these skills at another job?
Of course, I would! The better question is would I have been able to learn all these skills at another job and how long would that have taken? I didn’t work somewhere else, but I keep in touch with my college classmates so I have some idea of the answers to these questions. Many of my classmates have learned a bit more about statistical modelling and programming than I did, but most did not develop the broader skills outlined above.
I imagine my experience working somewhere else would have most likely included me working under someone who started 6-24 months before I did who may or may not know how to do their job very well. This is not what it’s like working at APPRISE. I work with Jackie, David, or Dan every single day and most days I work with all three! To spend the first year of my career working with people who have decades of experience working in my field on a daily basis. This is a privilege very few people get to experience. Their doors are open about 75% of the time and it is rare they don’t have time to answer my questions immediately.
You don’t get this kind of access to this kind of expertise at your average workplace. Additionally, all my supervisors know my strengths, weaknesses, and what skills I’m focusing on at any given moment so they can adjust their feedback accordingly. As stated above, I have spent a lot of time working on writing better emails. Dan sends me emails he writes so I can study how he structured a complicated topic. Jackie reviews my email drafts to help me focus on key points and readability. David asks me guiding questions to help me understand what was not included and helps me learn what information is necessary to include. This kind of individualized mentorship from multiple sources has been invaluable to my professional growth.
I would be remiss to only share the lessons I’ve learned from my supervisors when the rest of the staff at APPRISE have taught me so much as well. These lessons have covered subjects ranging from epistemology and how to write accurate data-cleaning programs to assessing the quality of New Jersey bagels and trying different types of pickles. We always help each other with our work, there’s no such thing as a stupid question, and we certainly find time to laugh along the way.
Tips for Working at APPRISE
If you’re hoping to work at APPRISE or you’re already here, here’s my list of tips I wish I knew when I arrived.
- If you’re not learning a lot, you are phoning it in and/or you aren’t taking initiative.
I know because I was doing both when I started. If you think no one at APPRISE can show you how to do something better, you’re wrong. There’s always room for improvement and feedback is always meant to ensure your work is of the highest quality. Regarding initiative, when you take ownership and show you’re thinking about next steps, more responsibility will follow. For me, this led to greater meaning and joy in my work. In short, “You get out of this experience what you put into it.”
- Keep a list of acronyms when you start.
There are a lot of acronyms at most policy research organizations, and you need to learn them quickly. I kept a notebook with a list of acronyms on the front when I started which paid off. Additionally, it’s helpful to identify the 5 most relevant acronyms you need to know for the projects you’re working on – it’s okay to ask your colleagues to identify the acronyms most relevant to your tasks.
- Do not walk into Jackie’s or David’s office without a pen and paper.
You will almost always need it. Every once in a while, I get overconfident and think I don’t need to bring a pen and paper. More often than not, I have needed to ask for a pen and paper.
- Ask questions.
If you don’t understand how something fits into the bigger picture, no one knows they need to explain it to you, and you WILL make bad decisions. Those bad decisions will confuse someone when they are checking your work, and everyone will wish you had asked your question. If you can’t think of the right questions to ask on the spot, it’s okay to come back five minutes later and ask your questions.
- Learn the “APPRISE way” before you disagree with it.
There are a lot of small habits we build into our processes that made absolutely zero sense to me when I started. However, there has not been a single step I initially disagreed with I don’t insist upon now. There’s a reason for everything we do. I would advise you to look for reasons to follow the habit and not a reason to not follow the habit – you will learn a whole lot more.
- Go to lunch with your colleagues.
Taking a short break allows you to clear your mind and helps you foster relationships which lead to a more collaborative culture. You will also learn when someone not working on a certain project has already spent two years learning how to solve your current problem in college. You can’t do this work alone and friendly relationships have made my job a lot easier. Maybe you’re smart enough to do this alone, but I certainly am not.
Blake Hannagan – Senior Policy Analyst