The NY Times columnist Ross Douthat recently wrote an op-ed on the untimely death of Christopher Hitchens. In the op-ed, Douthat mostly discusses how Hitchens was able to maintain a warm and cordial relationship with people, such as William Lane Craig, who hold remarkably nasty opinions and parade them as moral teachings. I fully agree with Douthat that this was one of the things which made Hitchens so powerful and it's an approach that's worth cultivating. If you can't calmly discuss the ethical and logical failures of your opponent's position with them, you're not doing yourself any favors.
Tuesday, December 20, 2011
Friday, December 16, 2011
Tuesday, December 13, 2011
Bayesian Balls
I've been working broadening my understanding Bayesian thinking using Christian Robert's book The Bayesian Choice and one of the early examples in the book had me confused. The example is 1.2.2 which goes like so:
When I was first exposed to Bayesian statistics, I was excited about the flexibility for fitting models. While I was aware that accepting these things called prior distributions had some epistemological implications, it did not bother me. I viewed (and still view) the specification of prior belief into a distribution of parameters as a valuable way of making the researchers thoughts on the problem explicit. After all if your analysis says what you want it to, but you can't justify your prior distribution to your peers, you won't have much luck convincing them of your result.
What I missed was that a prior distribution can be a way of conditioning on belief, experience, or observed events. This makes the prior distribution even more valuable because it can encode belief about a process rather than just a belief about the distribution. For example if one assumes that the interval \([0,1]\) is very long and the billard ball won't make it to the other end, some sort of decaying distribution on \(p\) would make more sense than the uniform. Robert briefly describes the rationale for using the prior distribution as follows:
A billard ball \(W\) is rolled on a line of length one, with a uniform probability of stopping anywhere. It stops at \(p\). A second ball \(O\) is then rolled \(n\) times under the same assumptions and \(X\) denotes the number of times the ball \(O\) stopped on the left of \(W.\) Given \(X\), what inference can we make on \(p\)?Robert states that the prior distribution on \(p\) is uniform on \([0,1]\). The first point of confusion for me was that the prior distribution in this example comes not from prior belief, but from the prior experiment where the ball \(W\) is rolled.
When I was first exposed to Bayesian statistics, I was excited about the flexibility for fitting models. While I was aware that accepting these things called prior distributions had some epistemological implications, it did not bother me. I viewed (and still view) the specification of prior belief into a distribution of parameters as a valuable way of making the researchers thoughts on the problem explicit. After all if your analysis says what you want it to, but you can't justify your prior distribution to your peers, you won't have much luck convincing them of your result.
What I missed was that a prior distribution can be a way of conditioning on belief, experience, or observed events. This makes the prior distribution even more valuable because it can encode belief about a process rather than just a belief about the distribution. For example if one assumes that the interval \([0,1]\) is very long and the billard ball won't make it to the other end, some sort of decaying distribution on \(p\) would make more sense than the uniform. Robert briefly describes the rationale for using the prior distribution as follows:
... the use of the prior distribution is the best way to summarize the available information (or even lack of information) about this parameter as well as the residual uncertainty.I remember this point from my mathematical statistics course but it's not surprising that it didn't strike me as especially important when I was dazzled by the fact that I just had to come up with a (log) posterior and a (log) prior and learn a couple of (simple!) algorithms to get answers out of my data.
Columbia Workshop: Samuel Kou
This post is very much about transtitions.
Samuel Kou's talk was about a very important applied biology problem: how does one predict the three-dimensional structure of a protein from it chemical structure. This is an interesting problem because it is quite important and initially you would think that science, in its current state, could provide some pretty good general answers.
If we could get the three-dimensional structures of proteins in a quick way from their chemical structure it would be a great practical advance. For example there are a number of protein mis-folding diseases, the most famous of which is Creutzfeld-Jakob disease and it's bovine cousin, "mad cow" (Bovine Spongiform Encephalopathy, BSE)[1]. Understand the structure of a wide variety of the prions (the mis-folded proteins) would definitely inform our understanding of how these diseases progress and perhaps how to interfere with that progression. We could study the transitions triggered by prions which cause normal proteins to take on the pathological shape. Carrying out the studies in silico would give us the opportunity to look for molecules which might interfere with the mis-folding process.
Conceptually, the road from knowing the chemical structure of a molecule made up of branching chains to knowing how it folds in on itself to get a 3-D structure is pretty easy. Let's say you mix a little oil with a little more water, shake it up a little and watch what happens. First the oil will be in small drops, then the drops will start to rise and their density nearer to the surface will increase (oil floats in water). As the oil drops come in contact, they will sometimes merge into a bigger oil drop (once there's a bit of contact between two oil drops in water what you have is a single very oddly shaped oil drop. Drops tend to a spherical shape, so the single oddly shaped drop gets reshaped to minimize the amount of contact with the water.) It's like going downhill, to a lower energy state: a transition.
With atoms there are also some configurations that are more stable than others because, depending on which atoms are next to each other, you get a lower or higher energy. Things tend to go to a lower energy (e.g.-people fall from ladders). Physical chemists have gotten pretty good at calculating the energy of a certain configuration of atoms[2] so once we have two alternate structures for a protein we can figure out which one will be favored. Bam, problem solved... sort of.
The trouble with big proteins is that we can't just compare two alternate structures. We would have to compare a lot of them. I think the key result of combinatorics for the practicing scientists is that once you start dealing with the number of possible combinations of \(k\) objects, call it \(N\), the number \(N\) gets very big very fast. If you could enumerate all the possible combinations, transform those to the x/y plane, and plot the energy of each configuration on the z axis, you would get something like figure 1.
Figure 1: big ol' molecule, location on x/y plane indicates the configuration, z axis indicates energy. The lowest energy configuration is at the bottom of the pit. Graphic from [3].
That well at the very bottom is the lowest energy configuration: it's the bottom of the hill. One way to avoid the enumeration problem is to take some starting configuration and instead of comparing it to all other possible ones, just compare it to some similar ones and choose the best configuration. Then repeat. Ad nauseum. With some luck (e.g.-you don't get stuck in a local minimum), and after enough time, the answer will be the lowest energy configuration. So how do you guess at similar shapes to define those possible transitions?
Kou called his solution the called the Fragment regrowth Energy-guided Sequential Sampler (FRESS)[4] which uses information about known protein structures from protein data bank (hurrah for open data!) as well as a few other tricks to choose a fragment of the current structure, delete it, and replace the atoms into the chain in the same order but in slightly different locations. Based on the talk I gathered that it works well and I can see why. It combines multiple sources of information about what combinations are likely, it modifies small parts of the molecule at a time, and it efficiently deals with the oddities of the sample space. It's an amazing piece of work, but this post isn't about that. It's about transitions.
What I wanted to write about is this: Kou stated that the goal of the sampler is to find the bottom of the well, the lowest energy state, because that's the conformation that the protein takes on in nature. Other statisticians in the room wondered at this property of protein folding in nature, one even (jokingly?) brought up God[5]. This issue came up because one of the key difficulties with a sequential sampler like Kou's is that if it's only considering a small neighborhood in protein-conformation space around the current conformation it might easily get stuck in a local minimum. It would never find the true global minimum. The same could be true in nature: either some or all of the proteins folded in an organism might end up at either one or several local minima.
In nature I can think of two things which influence how, starting from the unfolded state, a protein tries out new configurations. The first is random motion, and the neighborhood around the current configuration explored by this motion is controlled primarily by the temperature. We know typically know the temperature range a protein needs to fold in, and we can probably calculate what size of moves it makes. The second factor is a class of proteins called chaperone proteins (chaperonins). These proteins bind temporarily to an unfolded protein guide the folding process. The interaction results in a faster exploration of a particular part of protein-conformation space than would otherwise be possible. It might overcome barriers to folding which would otherwise be insurmountable. It's not hard to imagine that a chaperonin might lead another protein to folding into a local minimum configuration. As long as the local minimum was deep enough that the protein was trapped there at its usual temperatures, it could function without ever folding to its global minimum. I don't really know enough about this area to know whether the majority of proteins go to a local or global minimum, but if it hasn't been addressed before it would be an interesting question from an evolutionary perspective[6].
Thinking of prions and transitions again, the healthy and pathological shape of the protein are not thought to be chemically different. It's possible that the only difference is the conformation. In that case, it would be very interesting to know which one is the local versus global minimum conformation, and what folding steps are involved in the transition. This is actually the sort of question which might be addressed by FRESS (or any other sequential sampler for protein folding) which made me very excited to hear about the details of the algorithm and see how much of an improvement it was on the state of the art.
[1] Lovely, accessible post on both @LSF
[2] Yes, I am being charitable.
[3] http://media.caltech.edu/press_releases/13230
[4] http://jcp.aip.org/resource/1/jcpsa6/v126/i22/p225101_s1?isAuthorized=no
[5] To be fair, Kou did state the as far as he knew the structures confirmed by x-ray crystallography have all been global minima (though that's hard to verify).
[6] and it might even help in figuring out good proposal distributions for a sampler like FRESS.
Samuel Kou's talk was about a very important applied biology problem: how does one predict the three-dimensional structure of a protein from it chemical structure. This is an interesting problem because it is quite important and initially you would think that science, in its current state, could provide some pretty good general answers.
If we could get the three-dimensional structures of proteins in a quick way from their chemical structure it would be a great practical advance. For example there are a number of protein mis-folding diseases, the most famous of which is Creutzfeld-Jakob disease and it's bovine cousin, "mad cow" (Bovine Spongiform Encephalopathy, BSE)[1]. Understand the structure of a wide variety of the prions (the mis-folded proteins) would definitely inform our understanding of how these diseases progress and perhaps how to interfere with that progression. We could study the transitions triggered by prions which cause normal proteins to take on the pathological shape. Carrying out the studies in silico would give us the opportunity to look for molecules which might interfere with the mis-folding process.
Conceptually, the road from knowing the chemical structure of a molecule made up of branching chains to knowing how it folds in on itself to get a 3-D structure is pretty easy. Let's say you mix a little oil with a little more water, shake it up a little and watch what happens. First the oil will be in small drops, then the drops will start to rise and their density nearer to the surface will increase (oil floats in water). As the oil drops come in contact, they will sometimes merge into a bigger oil drop (once there's a bit of contact between two oil drops in water what you have is a single very oddly shaped oil drop. Drops tend to a spherical shape, so the single oddly shaped drop gets reshaped to minimize the amount of contact with the water.) It's like going downhill, to a lower energy state: a transition.
With atoms there are also some configurations that are more stable than others because, depending on which atoms are next to each other, you get a lower or higher energy. Things tend to go to a lower energy (e.g.-people fall from ladders). Physical chemists have gotten pretty good at calculating the energy of a certain configuration of atoms[2] so once we have two alternate structures for a protein we can figure out which one will be favored. Bam, problem solved... sort of.
The trouble with big proteins is that we can't just compare two alternate structures. We would have to compare a lot of them. I think the key result of combinatorics for the practicing scientists is that once you start dealing with the number of possible combinations of \(k\) objects, call it \(N\), the number \(N\) gets very big very fast. If you could enumerate all the possible combinations, transform those to the x/y plane, and plot the energy of each configuration on the z axis, you would get something like figure 1.
That well at the very bottom is the lowest energy configuration: it's the bottom of the hill. One way to avoid the enumeration problem is to take some starting configuration and instead of comparing it to all other possible ones, just compare it to some similar ones and choose the best configuration. Then repeat. Ad nauseum. With some luck (e.g.-you don't get stuck in a local minimum), and after enough time, the answer will be the lowest energy configuration. So how do you guess at similar shapes to define those possible transitions?
Kou called his solution the called the Fragment regrowth Energy-guided Sequential Sampler (FRESS)[4] which uses information about known protein structures from protein data bank (hurrah for open data!) as well as a few other tricks to choose a fragment of the current structure, delete it, and replace the atoms into the chain in the same order but in slightly different locations. Based on the talk I gathered that it works well and I can see why. It combines multiple sources of information about what combinations are likely, it modifies small parts of the molecule at a time, and it efficiently deals with the oddities of the sample space. It's an amazing piece of work, but this post isn't about that. It's about transitions.
What I wanted to write about is this: Kou stated that the goal of the sampler is to find the bottom of the well, the lowest energy state, because that's the conformation that the protein takes on in nature. Other statisticians in the room wondered at this property of protein folding in nature, one even (jokingly?) brought up God[5]. This issue came up because one of the key difficulties with a sequential sampler like Kou's is that if it's only considering a small neighborhood in protein-conformation space around the current conformation it might easily get stuck in a local minimum. It would never find the true global minimum. The same could be true in nature: either some or all of the proteins folded in an organism might end up at either one or several local minima.
In nature I can think of two things which influence how, starting from the unfolded state, a protein tries out new configurations. The first is random motion, and the neighborhood around the current configuration explored by this motion is controlled primarily by the temperature. We know typically know the temperature range a protein needs to fold in, and we can probably calculate what size of moves it makes. The second factor is a class of proteins called chaperone proteins (chaperonins). These proteins bind temporarily to an unfolded protein guide the folding process. The interaction results in a faster exploration of a particular part of protein-conformation space than would otherwise be possible. It might overcome barriers to folding which would otherwise be insurmountable. It's not hard to imagine that a chaperonin might lead another protein to folding into a local minimum configuration. As long as the local minimum was deep enough that the protein was trapped there at its usual temperatures, it could function without ever folding to its global minimum. I don't really know enough about this area to know whether the majority of proteins go to a local or global minimum, but if it hasn't been addressed before it would be an interesting question from an evolutionary perspective[6].
Thinking of prions and transitions again, the healthy and pathological shape of the protein are not thought to be chemically different. It's possible that the only difference is the conformation. In that case, it would be very interesting to know which one is the local versus global minimum conformation, and what folding steps are involved in the transition. This is actually the sort of question which might be addressed by FRESS (or any other sequential sampler for protein folding) which made me very excited to hear about the details of the algorithm and see how much of an improvement it was on the state of the art.
[1] Lovely, accessible post on both @LSF
[2] Yes, I am being charitable.
[3] http://media.caltech.edu/press_releases/13230
[4] http://jcp.aip.org/resource/1/jcpsa6/v126/i22/p225101_s1?isAuthorized=no
[5] To be fair, Kou did state the as far as he knew the structures confirmed by x-ray crystallography have all been global minima (though that's hard to verify).
[6] and it might even help in figuring out good proposal distributions for a sampler like FRESS.
Subscribe to:
Posts (Atom)