Black box variational inference for gammas

Whoa, this has been a long blogging hiatus for me.  I have no excuses other than I've been enjoying life and working hard.  So not excuses, reasons.

I return with a super light-and-fluffy post to share a guide to black box variational inference for gamma-distributed latent variables.  BBVI is very powerful, but I was having trouble applying it to gamma variables, so I asked Rajesh (its creator) for some tips.  I wrote the guide to try out his tricks on a very simple model and share them with other folks that might be having similar issues.  Have fun, ya'll.


the fastest way to embed fonts in a PDF (for me)

I needed to submit a PDF today with all fonts embedded.  I'm working on a Mac with TexShop, and after much angsty Googling, I found lots of answers that seemed to work for other people, but for one reason or another, weren't working for me.

This is what I ended up doing: I created the PDF like usual with TexShop (without all the fonts embedded...I'm looking at you, Helvetica).  Then, I used pdf2ps to convert it to a postscript file, and then Adobe Distiller to convert back to a PDF (with all fonts embedded).  There may be other ps to pdf options that do the font embedding (Dstiller isn't free), but I was sure Distiller did, and I was on a timeline.

This is the PDF equivalent of "just reboot it."  It's silly that embedding fonts is such a messy process.


An Insider's Spain

Way back in May, a week after I submitted a paper (which got in!) and a two days after Nathaniel gave his final oral presentation for his dissertation, we headed off to Spain!  It was a very different sort of vacation, as Nathaniel grew up in Madrid and has an insider's perspective and also lots of friends still living there.

The first day, Nathaniel was actually a little quiet—memories started flooding in as soon as we hit the Madrid airport, and included things as small as being on a particular ramp or hearing the recorded metro voice.  We spent the afternoon walking in the Retiro park, eating bocadillos, looking the book shops, street vendors and performers, and just generally people watching.  Dinner was at Casa Mingo, where Nathaniel's family used to go regularly.

This one is for my mom, the Iris junkie.
The following day, we saw the royal palace, walked through the downtown, saw the museo del Prado, and visited the botanical gardens in Retiro, which had an abundance of irises and other flowers in bloom.  In the evening , we had dinner with a bunch of Nathaniel's friends from school.  It was fun to meet them, joke around, and just have a relaxing evening. Sunday was more memories: we visited Nathaniel's old church congregation, walked by his old apartment and school, and met with another dear friend.

The following day, we left Madrid, which began a very different phase of the trip.  We did some hiking and drove down to Toledo.  There, we saw the El Greco museum, which has both art and living spaces restored to late 16th century style.  We also saw a cathedral, tapestry museum, Jewish museum, and an Alcazar which had been converted to the most elaborate military museum I've ever seen.  Toledo also has a marzipan factory, so I had to put some in my mouth.

Then, off to Córdoba!  After dinner, we had tea and sweets in cute little Moroccan tea shop.  The following dat we saw the Mezquita, which was thoroughly strange.  It was once a mosque, but a cathedral had since been built in the center, and services were going on while we were there.  The guide brochure had a strong catholic bias, with only one of six panels devoted to its Muslim history, most of which read a little bit like "it was a church, and then the Muslims tore that down and built a mosque, and so we feel totally justified in making it a cathedral, because it was once a church.  Do you hear that?  We were here first."  I exaggerate, of course, but it was very defensive.

Mosque-cathedral hybrid. Just a cute alley we found.

We also saw a roman water wheel, and another alcazar, which was a child's dream fort with lovely garden and a fun irrigation system.  I would let you guess who was dorking out about what, but the pictures give it away.

Nathaniel dorking aout about irrigation. Me dorking out about flowers.

Next up was Granada.  We arrived in the evening, caught a flamenco show, and watched the sun set over Alhambra.  We got up early the next day because Alhambra tickets sell out quickly, and that was the major reason we were there.  Thankfully, we got to see it, and it was both amazingly elaborate and extensive.  When I wasn't cooing over the gardens, I was on a mission to find all 17 wallpaper groups, which I'll post about later.  After we spent most of the day in Alhambra, we took a quick hike and then headed over to Huelva, where Nathaniel had another classmate (Charli) we were going to stay with.

Orange and cream colored pomegranate blossoms.

We used Huelva as a launching point to see Sevilla.  There we saw yet another cathedral and alcazar; the latter we recently recognized in a Game of Thrones episode (Dorne).  We also saw an amazing face-off between a duck and a peacock.  The duck was apathetically looking for food while the peacock was aggressively shaking its feathers at the duck.  It went on for about 10 minutes before we moved on.  Eventually we joined up with our hosts Charli and Laura who encouraged us to do silly touristy things.  That evening, it was Charli's goal to present me with food I would refuse.  We began with caracoles (snails), then proceeded to tiny clams, a large octopus tentacle, and finished with little squid.  I ate everything, but by the time I got to the squid, I was very full, so only had one.

Charli's idea.

After departing from Charli and Laura's place the next morning with bellies full of fresh churros dipped in hot chocolate, we made a quick stop at the replicas of Christopher Columbus's ships, and then headed to Segovia.  There, we marveled at the massive Roman aqueduct and had dinner at the famous (both within the Chaney family and without) Restaurante Jose María, which lived up to every expectation.  Jose María himself even came by our table to ask how things were.

For some reason, every statue of Christopher Columbus we
saw has him pointing.

Ordesa was up next—it was a bit of a drive to the park, as it is on the French border.  There we spent three nights up in the mountains.  We saw beautiful wildflowers, and families of chubby marmots and sprightly mountain goats.  The rock formations were simply stunning—the beauty of Ordesa cannot properly be put into words.  I'll just put up a bunch of pictures instead.

Roman bridge.
This waterfall fascinated Nathaniel.
It came out of the side of the rock.

Looking hardcore with my ice pick.
Despite the snow, it was hot.
And because of the snow it was bright.
So we build this shelter for lunch
after deciding an igloo would take too long.
The gap tooth in the background is a pass,
with France on the other side.
His turn to look hardcore.
But I needed the last word in hardcoreitude.

After hiking, we made our way down to Barcelona, which felt far more foreign than I expected, probably due to minor language barriers.  We had a lovely evening with Nathaniel's brother, who was there on business, then saw the old city and the Sagrada Familia the following day.  At first I was hesitant to go inside the basilica, as it reminded me of a drippy sand castle, but inside, the architectural mastery was shown to its fullest, as the colored glass windows cast fantastic shadows over smooth geometric shapes.  The day after, we saw Castell de Montjuïc and the area surrounding the Olympic grounds.

Like any long trip, I was ready for the routine of home at the end, but I was less exhausted than trips usually make me.  I think this was due to our slower-than-usual pace and much of it being already familiar to Nathaniel.  If we are blessed with the ability to do so, I would love to make Spain a regular vacation destination for us, returning every five or six years.  It has a piece of home.

Let's end with this awesome flower we saw in Barcelona.
Two petals are each half pink and half white.


What does it mean to be a millennial?

There's a problem with labels.  They make the world simpler, easier to parse and understand, but the problem is that all of the stereotypical associations with a label are rarely true for every application of it.

Take gender for instance.  It is absolutely correct to call me a woman, but what baggage does that term bring?  What extra information does it give you when I accept that label?  If you had to guess the length of my hair, it would likely be longer than if you were to guess the same for a man.  If you were to first meet me, what do you think you would assume my profession to be?  My hobbies?  My favorite movies?  Labels help us guess these things, and you might be right on several of them, but not on all of them.  These guesses also help us ask right questions so we can learn more about people and move beyond the labels.  If all you needed to know was my gender to know everything about me, it would be a tragically simple world.

I want to talk about the label applied to people of my generation: millennial.  There are some ways in which I am absolutely a millennial, and other ways in which I wish I could distance myself from the term as much as possible.

In particular, I identify strongly with the tech-savvy elements of the label.  My childhood was also associated with world events and media particular to my generation, both of which I feel neutral about.  What I wish to avoid is the label of being selfish and indulgent.  While everyone has their moments, this is not something I want to accept for myself.  This is to say, I may be more self-centered than those older than me, but I don't want to be and I'm anxious to work on it.  Also, maturity just comes with age.  It's hard to know what the final millennial legacy will be while we are still so young.

To work on changing the associations with this label, two things need to happen.  First and foremost, we just need to buck up and start thinking about others.  We need to ask: how can I be a better child, romantic partner, or employee?  How can I be a more polite stranger in stores, on the street, in my car, and online?  How can I contribute to the world?  And when we ask this final question, the emphasis should be on how the world can be improved, not on our role in improving it.  These are good ideas for anyone to think about, and many millennials are already asking these things, but it seems like the cultural expectations for us are lower than for previous generations, which brings me to my second point.

Older generations need to expect more from us.  Don't coddle me: I want to be told when I'm not up to snuff.  And while you don't need to be mean, I need you to be honest.  My feelings will inevitably be hurt, just because change is inherently painful, but it is worth the effort on both our parts.  I need to be told when to put away my electronics, when I need to work harder, and when I'm being rude.  I need to be told not because I'm sluffing the responsibility off on you, but because sometimes I just need to know my behavior is wrong in order to change it.

Also: keep your praise mild.  I'd rather feel like there's room for improvement than like I'm the best thing ever.  The humility that comes from feeling like you're never good enough is far better than the arrogance that comes from feeling like everything you do is golden.  Err on the side of being hard on me: excellence comes from being pushed.

To tie the two points together: we (millennials...and everyone, really) also need to learn how to accept criticism.  This means we need to value results more than how we feel about our work to produce them.  This and basically all issues of selfishness come down to priorities. We just need to move ourselves lower down in the list.

It doesn't matter if you're a millennial or not, these ideas apply to everyone, including myself.  Everyone, no matter how selfless they are, should consider how to improve.  We can improve how we give and receive advice.  We can improve how we treat others.  We can improve our priorities.  And maybe a few rare people should decide that they need to prioritize themselves more, but chances are that those people aren't reading this (spending their time, instead, on others).  My only hope is that our generation will have its fair share of those individuals.  There is still time to write our legacy.  Let's make it a generous one.


No explanation needed

Just read this amazing article that talks about socialized male speech dominance, which relates to who interrupts whom and how women are often just blatantly ignored.
A woman, speaking clearly and out loud, can say something that no one appears to hear, only to have a man repeat it minutes, maybe seconds later, to accolades and group discussion.
Yup.  I've experienced this exact dynamic, though thankfully not with great frequency.  An example along these lines, from the article:
Solnit's tipping point experience really did take the cake. She was talking to a man at a cocktail party when he asked her what she did. She replied that she wrote books and she described her most recent one, River of Shadows: Eadweard Muybridge and the Technological Wild West. The man interrupted her soon after she said the word Muybridge and asked, "And have you heard about the very important Muybridge book that came out this year?" He then waxed on, based on his reading of a review of the book, not even the book itself, until finally, a friend said, "That's her book." He ignored that friend (also a woman) and she had to say it more than three times before "he went ashen" and walked away. If you are not a woman, ask any woman you know what this is like, because it is not fun and happens to all of us.
Another recent related example: about two or three weeks ago, I was sitting on the couches in my church building working on my laptop since NWC had to be there early for a meeting.  The wifi was down, so I opened a new tab in my browser and it gave me the typical can't-find-the-network page.  A male missionary caught a glance of my screen and waltzed over, asking if I was having computer problems.  I told him that the wifi was just having issues and that I was fine.  He proceed to tell me that he didn't have that much experience with Macs (ignoring my statement that I didn't need help), so I said I'm a computer scientist; I think I can handle it.  After all that, he still acts like he hasn't heard a word I said and says something along the lines of sorry I couldn't help you.  How about sorry I assumed you needed help with a computer because you're a woman instead?  Because I'm pretty sure he never would have done the same thing to a man my age.

The author suggests that girls learn to say the following phrases more regularly: "Stop interrupting me," "I just said that," and "No explanation needed."  Yet none of these apply to my recent experience, so maybe we should add: "Thank you, but I don't need your help."


women in STEM: where to intervene

I am at workshop this week, and had an interesting sideline conversation today.  We both agreed that there was culture-induce inequality of the genders in STEM fields, but the fellow I was talking with disagreed with with me on how to fix things.

I was saying if a department had zero female faculty, then they should make an effort to encourage women to apply.  I was not saying that they should be given less stringent requirements or that a lesser qualified female candidate should be preferred over a more qualified male, but that they should simply make an effort to seek out qualified candidates.

He was arguing that we should address the cultural influence not at the faculty-hiring level, but at the elementary school level.  While I think that there is much that can be done in elementary school, this would never be enough.  A girl could be taught to love math in the third grade, but by the time she made it through the gauntlet of media aimed at middle schoolers, there is very little chance that the elementary school efforts could outweigh other influences.

If we're going to make an effort to level the playing field for men and women in STEM, it needs to happen at every level.  We cannot focus only on faculty or other employment nor can we only dedicate our efforts to primary or secondary education.  The efforts go hand in hand: without role models, girls may think that certain jobs are unappealing.  Without efforts in schooling, there will not be a population of women with the interests and tools to pursue STEM fields.

But these aren't the only areas that influence girls' choices to pursue math-y fields or not.  TV shows, movies, toys, magazines, advertisements, parenting choices, religious communities, political parties, news, and books all shape the way we think about ourselves, how we contextualize our existence in the world.

Any time a mother says to a daughter: I'm not good at math, ask your dad, the daughter's expectations  are adjusted; it's now okay not to be good at math.  Worse yet, she's taught that it's okay to deprecate yourself, which lowers self-esteem and propagates negative cultural influences.  Now, it's totally legitimate for many women not to enjoy math or not be skilled at it, but for the sake of your daughter, your own skill doesn't need to enter into it.  Instead, a mother could say: Oh!  This is important, but it's your dad's responsibility to help with math homework.

Since the influences are so broad and diverse, what can we possibly do?  When do we intervene and how much?  In most cases, I don't think we need a broad campaign; we just do what we can where we can.  If we're on a hiring committee, we seek out qualified candidates.  If we're an elementary school teacher, we make sure to present the material without gender (or other) biases, and teach all students that they can excel.  If we're a manager, we assign tasks simply based on the ability to perform them.  This applies to all areas of bias, be they gender, ethnic, orientation, religious, or something else.

The idea here is that popular culture and media pushes individuals in a certain direction: women need to conform to certain standards of beauty, families need to be constructed a certain way, boys should be good at math and sports.  When we see something that feels wrong, we just need to push back in whatever ways we can and hope that our efforts offset the influences we deem to be negative.

The frustrating thing about the conversation today was what felt like a willful misunderstanding.  He asked things like, So do you think that a certain percentage of the faculty in a department should be female?  No; it's silly to set numbers in advance.  If things are grossly unbalanced (like the specific case I was actually talking about with zero female faculty in a reasonably large department), then the department should make an effort to hire women.  He then objected that you shouldn't fight an inequality with the inequality of hiring under-qualified women.  What bothers me about this argument is that it assumes there aren't qualified women; there are, you just have to look a little harder for them, which is what I was proposing.  He then shifted the focus to contrasting elementary school initiatives vs. faculty hiring efforts and eventually even went so far as to compare recruiting females to recruiting people with blue eyes in an effort to question what constitutes diversity.

I think it's common for men to feel threatened by the idea that women should be more actively recruited.  It's natural: it implies that they're less desirable, which isn't the case.  The reason I bring this up isn't to shame a particular person (I intentionally included no names), but to highlight an area that can be improved.  I don't think everyone should agree with me, and I do think that he's right in some ways: women are just people and should be treated as such.  The problem is, that until they're actually treated as people in all facets, we can't pretend that everything will just be okay if teach elementary kids to like math and science.  There are so many factors at play that we need to make an effort to change culture; it doesn't happen naturally.

And in the effort to change to that culture, I think we need to be a little more careful about they way we converse about these issues.  I know I get defensive and other people do too; I think this is the first thing that can change, because as soon as it does, we can stop talking past each other and actually get some stuff done.


LDS Public Affairs

This past Sunday, I received and accepted a new calling: Secretary to the (horrible acronym warning) NYMMPAC, which stands for New York Metro Multistake Public Affairs Council.  It covers 3 Coordinating Councils, or 13 Stakes and 131 units.

Generally speaking, the idea of church public affairs makes me a little queasy.  In an ideal world, the good works of the church should speak for themselves and we should have no need to influence public opinion of us.  That said, I think the public affairs groups aren't there solely to bring about good press—the director emphasized to me that "public affairs" is really an unfortunate name, since there's much more to it than that.  But, prior to this, my perception of Public Affairs was the folks who force members to wear yellow Helping Hands vests and do photographed service activities.  Or, more recently, PA was represented by the token woman that introduced the Apostles for the press release on religious freedom.

So why did I accept the calling?  Part of it was certainly ego.  But another big part of why I accepted is that I was recommended for the position because of my strong opinions.  This means that they knew what they were getting, and I had license to be myself.  I also warned them that I have a blog.

I have a friend at church who is on the NYMMPAC, and she and I have had regular hallway discussions on church issues (in lieu of attending Sunday School).  She acknowledges that some aspects of Public Affairs and popular LDS culture need to be improved, and has been working to make things better.  The idealistic optimist in me says: now I can be a part of that effort!

While I do need to keep some things confidential, I hope to keep a record of some of my public affairs experiences, to improve transparency, define the role of public affairs a little more concretely, and to prevent me from drinking too much cultural kool-aid.

Update: Since writing this post, lots of people have told me about great PA experiences, like:
Folks have also pointed out that PA is one of the few areas in the church hierarchy where women have impactful roles.  Now I'm feeling pumped.


ice skating as a waltz

Yesterday, NWC and I went ice skating on the three frozen bodies of water in our area—a tour of the ponds and lake.  The muscles in my feet usually get tired after about 40 minutes, so the was a nice way to take advantage of the outdoor ice skating opportunity without wearing myself out.  As soon as we got tired, we took a break by switching to the next one.

I'm still very much a novice when it comes to ice skating.  This season I've made it my goal to practice two things: stopping and skating continuously (instead of two pushes and parallel gliding; repeat).  My biggest problem with smooth skating was speed; I could skate continuously by constantly pushing off one foot then the other, but then I end up going far too fast and losing control.

Yesterday I finally made a break through on skating continuously: I turned it into a waltz.   I was watching a slow experienced skater, and started to count how often she pushed off, and it was every third beat.  Once I found the rhythm and turned it more into a dance, it was like flipping a switch.  Now to figure out how to stop properly...


Allison's Law: "The mess has to go somewhere"

When I was a growing up, we had a standard of cleanliness in our house called "daddy-clean." My brother and I were asked regularly to clean are rooms, like most American children, but when we were done, mom would always ask: Is it daddy-clean?  This usually resulted in a second round of cleaning to make sure everything was out of sight.

There was a flaw to this paradigm, however, which was that daddy-clean only applied to things that were visible.  Thus, I learned the art of shoving everything under my bed, which had a convenient bedskirt to hide everything.  Toys, clothes, paper; everything went underneath.  When under-the-bed got full, the closet was my second choice.  Eventually my parents found out about this, due to an abundance of random objects poking out, but they allowed me my secret messes so long as they didn't get in the way of finding important things, which they occasionally did.

Nowadays my messes look a little different.  In addition to paper, I have more abstract things like source code.  And my experience growing up has taught me: the mess has to go somewhere.  Most of the time, this is just a trade off between different aspects of cleanness of an end product and time, but it applies in so many cases.

Consider the process of creating a user interface.  The mess can go into the source code; everything hacked together in an ugly mess underneath.  The mess could also go into the UI itself: bad design with beautifully easy implementation.  Or, the mess could be absorbed with lots of time to have pretty code and sleek presentation.

Or consider a different piece of software, like an operating system.  The mess could go into the kernel, into the user experience, or passed on to developers for that platform.  Or, again, the mess can be absorbed by lots of time and effort.

In my experience, the mess of the very pretty Mac OS is passed on to developers.  D3, with its steep learning curve and beautiful graphics, also passes the mess to programmers.  Easy-to-use and powerful libraries like ggplot2 for R probably put the mess in some combination of the under-the-hood code and time.

I've also been thinking about this in terms of (machine learning) model development.  Usually elegant models require an intense amount of time to polish into their perfected forms.

It's not always the right choice to absorb mess with time; sometimes a project isn't worth doing exceptionally cleanly.  I think it is always worth it, however, to consider where your mess will be going in order to make a measured choice.


on lady tech events

I recently had a colleague ask me about women in tech events, and I wanted to adapt my response into a blog post.

Women in tech events and long-term mentoring can be very fulfilling, especially when the audience is narrow enough. That said, I'm always hesitant about female tech mentoring because it seems like there is so very much of it, both soliciting mentors and mentees, so it's easy to commit to more than you actually want to do.

It's important that every individual takes a step back asks themselves: How much time do I want to dedicate to networking and mentorship? What kinds of interactions are most valuable to me? (What have I enjoyed about other events I attended?) Plan the big picture first, and then use that as a roadmap to make the smaller choices.

In the end, these are personal choices about you and your career. For me, going to WiML every year is enough. But other people might want more support or networking. Still others might not care at all about things like this—again, this is totally personal. It also varies depending on where you are in your career, because our needs and preferences evolve with time.

As a one-off, these kind of things won't make or break you, but it's the aggregate of multiple events over your career. If you're not certain about a particular event, it might be worth going, just to see how fulfilling these kinds of events are to you, so you can make informed choices going forward.


evening art project

I've been trying to do more artistic things lately—it's not really a new year's resolution, but it stated during the holidays.  After a few frustrating half-finished things, I finally completed a piece this evening.  NWC wanted me to name him, and after he refused both Seahorse and Seahorsie, I went with Herbert



How to contact an academic

Ever since I started graduate school, I've gotten the occasional academic spam from applying students.  These folks are obviously trying to be clever by automating their emails, but usually people see through it.  I also regularly get emails from people who want to work for my advisor, or from friends and strangers who want advice on applying to grad school.  It feels as though I've repeating myself recently, so I'm collecting all of my advice about contacting professors (or other academics) into a blog post.

1. Keep it brief

Nobody likes a long rambling email, except maybe your mom.  At the most, attach or link a CV/resume.   A general template is to introduce yourself, express why you're interested in their work, and then tell them what you want.  Each of these can easily be one sentence, maybe two.  If you want to put lots of effort into this email because it's really important to you, don't just make it longer.  Instead, take the time to craft your words so they read quickly and easily.

2. Have a purpose / Make it easy for them to respond

Think about why you're writing to this person.  Is it because you want to be noticed in graduate applications?  If that's it, just skip the email because it probably won't do anything.  You should only email folks when you have a purpose.  Examples of good purposes: you want to be a research assistant, want to join their reading group, would like to sit in on a group meeting, or would like to schedule a meeting with them (e.g., to get their advice on a project).  If you make it clear what you want, and show that it isn't that much work for them (15m of their time or a two-line email response), then they're much more likely to respond positively.

Now, contrary to what I just said, you can still write to a professor even if you just want to be noticed for graduate school applications.  You just need an additional reason, like one of the examples I mentioned above.  You should simply say that you're applying, would love to join their research group, and then give them that extra something else that allows them to respond easily, like asking if you can stop in on their group meeting or if they could put you in touch with one of their current students.  Simply saying "I'm applying to grad school at your university" is usually a waste of everyone's time, yours included.

If you don't actually need them to respond, you can just say so, but sometimes it's implicit.  "I just wanted to thank you for your great work on ABC; I've used it for XYZ and thought you might be interested in seeing the results at <link>."  Regardless of whether or not you want a response, make it clear why you're writing to them so that they know what to with the email; this is just general email etiquette.

3. Do your homework

Read their papers.  Download and run their code.  This takes a bit of work, but you should only be contacting a handful of people anyway—probably under a dozen if you're applying to graduate school, or maybe three if you're looking for a research position at your current institution.  If reading their papers is intensely painful for you, then maybe you shouldn't be writing to them.

You also want to avoid looking silly.  You shouldn't be asking them questions that they've answered recently on their blog, or asking for their latest reading group paper that's posted on an easy-to-find publicly available schedule.

The last reason you want to do your homework is that you don't want to look like a bot.  Consider the difference between "I like your research in machine learning" and "I like your research on visualizing topic models".  The first could be sent to thousands of people, but the second could only be sent to a handful.  If the content of your email is sufficiently personal, it's obvious that you're a human being that might be worth responding to.


Keep your messages short, sincere, and specific.  These seem like good guides for general correspondence as well.