Spring 2018 Physics 441

Office hours
David Roundy: MF 1-2, 401B Weniger or upon request. My door is open when I'm in my office, and students are welcome to enter with questions.
The syllabus is here.
Thermal Physics by Kittel and Kroemer. The textbook is not required, but the course will follow the text reasonably closely.
Course notes
If you wish, you may download this entire website as an ebook, or as a PDF file.
Homework will be due in class on Wednesday of each week (but not the first week of class). You should be able to start each homework the week before it is due. See the syllabus for details on homework grading. You may use the solutions (or any other resource you wish to use) but at the end of each problem, please cite what resources you used (students you worked with, whether you looked at the solutions, etc). Note that verbatim copying from any source is plagiarism. I recommend not using the solutions.

Introduction and course philosophy

This is your second course in thermal physics. Energy and Entropy took a thermodyamics-first approach, with primary emphasis on how you could measure something, and only later introducing how you could predict it. I strongly support this approach, but it is not the most common approach to thermal physics.

I will teach this course in a more traditional order and approach, following the text of Kittel and Kroemer. This is the textbook that I used as an undergraduate. It's an excellent book, but very much uses a more mathematical theory-first approach than I prefer.

Since this is now your second course in thermal physics, this should balance things out. With your Energy and Entropy background, you should be able to make physical connections with the math more easily. By organizing this course in this different way, I hope to broaden and deepen your understanding of thermal physics, while at the same time showing you a wide range of different physical systems.

Brief review of Energy and Entropy


If you consider two identical systems taken together (e.g. two cups of water, or two identical cubes of metal), each thermodynamic property either doubles or remains the same.

An extensive property, such as mass will double when you've got twice as much stuff.
An intensive property, such as density will be the same regardless of how much stuff you've got.

We care about extensivity and intensivity for several reasons. In one sense it functions like dimensions as a way to check our work. In another sense, it is a fundamental aspect of each measurable property, and once you are accustomed to this, you will feel very uncomfortable if you don't know whether it is extensive or intensive.

How to measure things

Measure dimensions and compute it. (extensive)
Force per area. Can equalize if systems can exchange volume. (intensive)
Find something that depends on temperature, and calibrate it. Alternatively use an ideal gas. Equalizes when systems are in contact. (intensive)
Challenging... measure work and heat (e.g. by measuring power into resistor). (extensive) \[\begin{align}W &= -\int pdV\end{align}\]
(extensive) Measure heat for a quasistatic process and find \[\begin{align}\Delta S &= \int \frac{dQ}{T}\end{align}\]
Measure changes of one thing as the other changes, with the right stuff held fixed.

First Law (Energy conservation)

\[\begin{align} dU &= dQ + dW \end{align}\]

Second Law (Entropy increases)

\[\begin{align} \Delta S_{system} + \Delta S_{environment} \ge 0 \end{align}\]

Thermodynamic identity

\[\begin{align} dU &= TdS - pdV \end{align}\]

Thermodynamic potentials

Helmholtz free energy
\[\begin{align} F &= U - TS \\ dF &= dU - TdS - SdT \\ &= -SdT -pdV \end{align}\]
\[\begin{align} H &= U + pV \\ dH &= dU + pdV + Vdp \\ &= TdS + Vdp \end{align}\]
Gibbs free energy
\[\begin{align} G &= H - TS \\ &= U -TS+pV \\ dG &= dH - TdS - SdT \\ &= -SdT +Vdp \end{align}\]

Statistical entropy

Boltzmann formulation (microcanonical or for large \(N\)): \[\begin{align} S(E) &= k_B \ln g \end{align}\] Gibbs formulation (always true): \[\begin{align} S(E) &= -k_B \sum_{i}^{\text{all states}} P_i \ln P_i \end{align}\]

Boltzmann ratio

\[\begin{align} \frac{P_i}{P_j} &= e^{-\frac{E_i - E_j}{k_BT}} \\ P_i &= \frac{e^{-\frac{E_i - E_j}{k_BT}}}{\sum_j^{\text{all states}} e^{-\frac{E_j}{k_BT}}} \end{align}\]

Thermal averages

The average value of any quantity is given by the weighted average \[\begin{align} \left<X\right> &= \sum_i^{\text{all states}} P_i X_i \end{align}\]

Week 1: Gibbs entropy approach

There are two different approaches for deriving the results of statistical mechanics. These results differ in what fundamental postulates are taken, but agree in the resulting predictions. The textbook takes a traditional microcanonical Boltzmann approach.

This week, before using that approach, we will reach the same results using the Gibbs formulation of the entropy (sometimes referred to as the "information theoretic entropy"), as advocated by Jaynes. Note that Boltzmann also used the more general Gibbs entropy, even though it doesn't appear on his tombstone.

Microstates vs. macrostates

You can think of a microstate as a quantum mechanical energy eigenstate. As you know from quantum mechanics, once you have specified an energy eigenstate, you know all that you can about the state of a system. Note that before quantum mechanics, this was more challenging. You can define a microstate classically, and people did so, but it was harder. In particular, the number of microstates classically is generally both infinite and non-countable, since any real number for position and velocity is possible. Quantum mechanics makes this all easier, since any finite (in size) system will have an infinite but countable number of microstates.

When you have a non-quantum mechanical system (or one that you want to treat classically), a microstate represents one of the "primitive" states of the system, in which you have specified all possible variables. In practice, it is common when doing this to specify what we might call a "mesostate", but call it a microstate. e.g. you might hear someone describe a microstate of a system of marbles in urns as defined by how many marbles of each color are in each urn. Obviously there are many quantum mechanical microstates corresponding to each of those states.

Small White Boards
Write down a description of one particular macrostate.

A macrostate is a state of a system in which we have specified all the properties of the system that will affect any measurements we may care about. For instance, when defining a macrostate of a given gas or liquid, we could specify the internal energy, the number of molecules (or equivalently mass), and the volume. We need to specify all three properties (if we want to ask, for instance, for the entropy), because otherwise we won't have a unique answer. For different sorts of systems there are different ways that we can specify a macrostate. In this way, macrostates have a flexibility that real microstates do not. e.g. I could argue that the macrostate of a system of marbles in urns would be defined by the number of marbles of each color in each urn. After all, each macrostate would still correspond to many different energy eigenstates.

Probabilities of microstates

The name of the game in statistical mechanics is determining the probabilities of the various microstates, which we call \(\{P_i\}\), where \(i\) represents a microstate. I will note here the term ensemble, which refers to a set of microstates with their associated probabilities. We define ensembles according to what constraints we place on the microstates, e.g. in this discussion we will constrain all microstates to have the same volume and number of particles, which defines the canonical ensemble. Next week/chapter we will discuss the microcanonical ensemble (which also constrains all microstates to have identical energy), and other ensembles will follow. Today's discussion, however, will be largely independent of which ensemble we choose to work with, which generally depends on what processes we wish to consider.


The total probability of all microstates added up must be one. \[\begin{align} \sum_i^{\text{all $\mu$states}}P_i = 1 \end{align}\]

This may seem obvious, but it is very easy to forget when lost in algebra!

From probabilities to observables

If we want to find the value that will be measured for a given observable, we will use the weighted average. For instance, the internal energy of a system is given by: \[\begin{align} U &= \sum_i^{\text{all $\mu$states}}P_i E_i \\ &= \left<E_i\right> \end{align}\]

where \(E_i\) is the energy eigenvalue of a given microstate. The \(\langle E_i\rangle\) notation simply denotes a weighted average of \(E\). The subscript in this notation is optional.

This may seem wrong to you. In quantum mechanics, you were taught that the outcome of a measurement was always an eigenvalue of the observable, not the expectation value (which is itself an average). The difference is in how we are imagining performing a measurement, and what the size of the system is thought to be.

In contrast, imagine measuring the mass of a liter of water, for instance using a balance. While you are measuring its mass, there are water molecules leaving the glass (evaporating), and other water molecules from the air are entering the glass and condensing. The total mass is fluctuating as this occurs, far more rapidly than the scale can tip up or down. It reaches balance when the weights on the other side balance the average weight of the glass of water.

The process of measuring pressure and energy are similar. There are continually fluctuations going on, as energy is going back and forth between your system and the environment, and the process of measurement (which is slow) will end up measuring the average.

In contrast, when you perform spectroscopy on a system, you do indeed see lines corresponding to discrete eigenvalues, even though you are using a macroscopic amount of light on what may be a macroscopic amount of gas. This is because each photon that is absorbed by the system will be absorbed by a single molecule (or perhaps by two that are in the process of colliding). Thus you don't measure averages in a direct way.

In thermal systems such as we are considering in this course, we will consider the kind of observable where the average value of that observable is what is measured. This is why statistics are relevant!

Energy as a constraint

Energy is one of the most fundamental concepts. When we describe a macrostate, we will (almost) always need to constrain the energy. For real systems, there are always an infinite number of microstates with no upper bound on energy. Since we never have infinite energy in our labs or kitchens, we know that there is a practical bound on the energy.

We can think of this as applying a mathematical constraint on the system: we specify a \(U\), and this disallows any set of probabilities \(\{P_i\}\) that have a different \(U\).

Small Group Question
Consider a system that has just three microstates, with energies \(-\epsilon\), \(0\), and \(\epsilon\). Construct three sets of probabilities corresponding to \(U=0\).
I picked an easy \(U\). Any "symmetric" distribution of probabilities will do. You probably chose something like: \[\begin{equation*} \begin{array}{rccc} E_i\text{:} & -\epsilon & 0 & \epsilon \\ \hline P_i\text{:} & 0 & 1 & 0 \\ P_i\text{:} & \frac12 & 0 & \frac12 \\ P_i\text{:} & \frac13 & \frac13 & \frac13 \end{array} \end{equation*}\]
Given that each of these answers has the same \(U\), how can we find the correct set of probabilities are for this \(U\)? Vote on which you think most acceptable!
The most "mixed up" would be ideal. But how do we define mixed-up-ness? The "mixed-up-ness" of a probability distribution can be quantified via the Gibbs formulation of entropy: \[\begin{align} S &= -k\sum_i^{\text{all $\mu$states}} P_i \ln P_i \\ &= \sum_i^{\text{all $\mu$states}} P_i \left(-k\ln P_i\right) \\ &= \left<-k\ln P_i\right> \end{align}\]

So entropy is a kind of weighted average of \(-\ln P_i\).

The Gibbs entropy expression (sometime referred to as the information theory entropy, or Shannon entropy) can be shown to be the only possible entropy function (of of \(\left\{P_i\right\}\)) that has a reasonable set of properties:

  1. It must be extensive. If you subdivide your system into uncorrelated and noninteracting subsystems (or combine two noninteracting systems), the entropy must just add up. Solve problem 2 on the homework this week to show this. (Technically, it must be additive even if the systems interact, but that is more complicated.)

  2. The entropy must be a continuous function of the probabilities \(\left\{P_i\right\}\). Realistically, we want it to be analytic.

  3. The entropy shouldn't change if we shuffle around the labels of our states, i.e. it should be symmetric.

  4. When all microstates are equally likely, the entropy should be maximized.

  5. All microstates have zero probability except one, the entropy should be minimized.

The constant \(k\) is called Boltzmann's constant, and is sometimes written as \(k_B\). The text prefers to set \(k_B=1\) in effect, and defines \(\sigma\) as \(S/k_B\) to make this explicit. I will include \(k_B\) but you can and should keep in mind that it is just a unit conversion constant.

How is this mixed up?

Small Group Question
Compute \(S\) for each of the above probability distributions.
\[\begin{equation*} \begin{array}{rccc|c} E_i\text{:} & -\epsilon & 0 & \epsilon & S \\ \hline P_i\text{:} & 0 & 1 & 0 & 0\\ P_i\text{:} & \frac12 & 0 & \frac12 & \ln 2 \\ P_i\text{:} & \frac13 & \frac13 & \frac13 & \ln 3 \end{array} \end{equation*}\] You can see that if more states are probable, the entropy is higher. Or alternatively you could say that if the probability is more "spread out", the entropy is higher.

Maximize entropy

The correct distribution is that which maximizes the entropy of the system, its Gibbs entropy, subject to the appropriate constraints. Yesterday we tackled a pretty easy 3-level system with \(U=0\). If I had chosen a different energy, it would have been much harder to find the distribution that gave the highest entropy.

Small Group Question
Find the probability distribution for our 3-state system that maximizes the entropy, given that the total energy is \(U\).
We have two constraints, \[\begin{align} \sum_\mu P_\mu &= 1 \\ \sum_\mu P_\mu E_\mu &= U \end{align}\] and we want to maximize \[\begin{align} S &= -k\sum_\mu P_\mu \ln P_\mu. \end{align}\] Fortunately, we've only got three states, so we write down each sum explicitly, which will make things easier. \[\begin{align} P_- + P_0 + P_+ &= 1 \\ -\epsilon P_- + \epsilon P_+ &= U \\ P_+ &= P_- + \frac{U}{\epsilon} \\ P_- + P_0 + P_- + \frac{U}{\epsilon} &= 1 \\ P_0 &= 1 - 2P_- - \frac{U}{\epsilon} \\ \end{align}\] Now that we have all our probabilities in terms of \(P_-\) we can simplify our entropy: \[\begin{align} -\frac{S}{k} &= P_-\ln P_- + P_0 \ln P_0 + P_+\ln P_+ \\ &= P_-\ln P_- + \left(P_- + \tfrac{U}{\epsilon}\right)\ln\left(P_- + \tfrac{U}{\epsilon}\right) \notag\\&\quad + \left(1-2P_--\tfrac{U}{\epsilon}\right)\ln\left(1-2P_--\tfrac{U}{\epsilon}\right) \end{align}\] Now we can maximize this entropy by setting its derivative to zero! \[\begin{align} -\frac{\frac{dS}{dP_-}}{k} &= 0 \\ &= \ln P_- + 1 - 2\ln\left(1-2P_--\tfrac{U}{\epsilon}\right) -2 \notag\\&\quad + \ln\left(P_- + \tfrac{U}{\epsilon}\right) + 1 \\ &= \ln P_-- 2\ln\left(1-2P_--\tfrac{U}{\epsilon}\right) +\ln\left(P_- + \tfrac{U}{\epsilon}\right) \\ &= \ln\left(\frac{P_-\left(P_- + \tfrac{U}{\epsilon}\right)}{ \left(1-2P_--\tfrac{U}{\epsilon}\right)^2}\right) \\ 1 &= \frac{P_-\left(P_- + \tfrac{U}{\epsilon}\right)}{ \left(1-2P_--\tfrac{U}{\epsilon}\right)^2} \end{align}\] And now it is just a polynomial equation... \[\begin{align} P_-\left(P_- + \tfrac{U}{\epsilon}\right) &= \left(1-2P_--\tfrac{U}{\epsilon}\right)^2 \\ P_-^2 + \tfrac{U}{\epsilon}P_- &= 1 - 4P_- -2\tfrac{U}{\epsilon} + 4P_-^2+4P_-\tfrac{U}{\epsilon} + \tfrac{U^2}{\epsilon^2} \end{align}\] At this stage I'm going to stop. Clearly you could keep going and solve for \(P_-\) using the quadratic equation, but we wouldn't learn much from doing so. The point here is that we can solve for the three probabilities given the internal energy constraint. However, doing so is a major pain, and the result is not looking promising in terms of simplicity. There is a better way!

Lagrange multipliers

If you have a function of \(N\) variables, and want to apply a single constraint, one approach is to use the constraint to algebraically eliminate one of your variables. Then you can set the derivatives with respect to all remaining variables to zero to maximize. This is what you presumably did in the last activity. However, in many cases this isn't feasible. And when there are many variables, it is almost universally inconvenient. A nicer approach for maximization under constraints is the method of Lagrange multipliers.

The idea of Lagrange multipliers is to introduce an additional variable (called the Lagrange multiplier) rather than eliminating one. This may seem counterintuitive, but it allows you to create a new function that can be maximized by setting its derivative with to all \(N\) variables to zero, while still satisfying the constraint.

Suppose we have a situation where we want to maximize \(F\) under some constraints: \[\begin{align} F = F(w,x,y,z) \\ f_1(w,x,y,z) = C_1 \\ f_2(w,x,y,z) = C_2 \end{align}\] We define a new variable \(L\) as follows: \[\begin{align} L &\equiv F + \lambda_1 (C_1-f_1(w,x,y,z)) + \lambda_1 (C_2-f_2(w,x,y,z)) \end{align}\] Note that \(L=F\) provided the constraints are satisfied, since the constraint means that \(f_1(x,y,z)-C_1=0\). We then maximize \(L\) by setting its derivatives to zero: \[\begin{align} \left(\frac{\partial L}{\partial w}\right)_{x,y,z} &= 0\\ &= \left(\frac{\partial F}{\partial w}\right)_{x,y,z} - \lambda_1 \frac{\partial f_1}{\partial w} - \lambda_2 \frac{\partial f_2}{\partial w} \\ \left(\frac{\partial L}{\partial x}\right)_{w,y,z} &= 0\\ &= \left(\frac{\partial F}{\partial x}\right)_{w,y,z} - \lambda_1 \frac{\partial f_1}{\partial x} - \lambda_2 \frac{\partial f_2}{\partial x} \end{align}\] \[\begin{align} \left(\frac{\partial L}{\partial y}\right)_{w,x,z} &= 0\\ &= \left(\frac{\partial F}{\partial y}\right)_{w,x,z} - \lambda_1 \frac{\partial f_1}{\partial y} - \lambda_2 \frac{\partial f_2}{\partial y} \\ \left(\frac{\partial L}{\partial z}\right)_{w,x,y} &= 0\\ &= \left(\frac{\partial F}{\partial z}\right)_{w,x,y} - \lambda_1 \frac{\partial f_1}{\partial z} - \lambda_2 \frac{\partial f_2}{\partial z} \end{align}\] This gives us four equations. But we need to keep in mind that we also have the two constraint equations: \[\begin{align} f_1(x,y,z) &= C_1 \\ f_2(x,y,z) &= C_2 \end{align}\]

We now have six equations and six unknowns, since \(\lambda_1\) and \(\lambda_2\) have also been added as unknowns, and thus we can solve all these equations simultaneously, which will give us the maximum under the constraint. We also get the \(\lambda\) values for free.

The meaning of the Lagrange multiplier

So far, this approach probably seems pretty abstract, and the Lagrange multiplier \(\lambda_i\) seems like a strange number that we just arbitrarily added in. Even were there no more meaning in the multipliers, this method would be a powerful tool for maximization (or minimization). However, as it turns out, the multiplier often (but not always) has deep physical meaning. Examining the Lagrangian \(L\), we can see that \[\begin{align} \left(\frac{\partial L}{\partial C_1}\right)_{w,x,y,z,C_2} &= \lambda_1 \end{align}\]

so the multiplier is a derivative of the lagrangian with respect to the corresponding constraint value. This doesn't seem to useful.

More importantly (and less obviously), we can now think about the original function we maximized \(F\) as a function (after maximization) of just \(C_1\) and \(C_2\). If we do this, then we find that \[\begin{align} \left(\frac{\partial F}{\partial C_1}\right)_{C_2} &= \lambda_1 \end{align}\]

I think this is incredibly cool! And it is a hint that Lagrange multipliers may be related to Legendre transforms.

Maximizing entropy

When maximizing the entropy, we need to apply two constraints. We must hold the total probability to 1, and we must fix the mean energy to be \(U\). This time I'm going to call my lagrange multipliers \(\alpha k_B\) and \(\beta k_B\) so as to reduce the number of subscripts required and also to make all the Boltzmann constants go away. \[\begin{align} L &= S + \alpha k_B\left(1-\sum_i P_i\right) + \beta k_B \left(U - \sum_i P_i E_i\right) \\ &= -k_B\sum_iP_i\ln P_i + \alpha k_B\left(1-\sum_i P_i\right) \notag\\&\quad + \beta k_B \left(U - \sum_i P_i E_i\right) \end{align}\] where \(\alpha\) and \(\beta\) are the two Lagrange multipliers. I've added here a couple of factors of \(k_B\) mostly to make the \(k_B\) in the entropy disappear. We want to maximize this, so we set its derivatives to zero: \[\begin{align} \frac{\partial L}{\partial P_i} &= 0 \\ &= -k_B\left(\ln P_i + 1\right) - k_B\alpha - \beta k_B E_i \\ \ln P_i &= -1 -\alpha - \beta E_i \\ P_i &= e^{-1 -\alpha - \beta E_i} \end{align}\] So now we know the probabilities in terms of the two Lagrange multipliers, which already tells us that the probability of a given microstate is exponentially related to its energy. At this point, it is convenient to invoke the normalization constraint... \[\begin{align} 1 &= \sum_i P_i \\ &= \sum_i e^{-1-\alpha-\beta E_i} \\ &= e^{-1-\alpha}\sum_i e^{-\beta E_i} \\ e^{1+\alpha} &= \sum_i e^{-\beta E_i} \\ \end{align}\] Where we define the normalization factor as \[\begin{align} Z \equiv \sum_i^\text{all states} e^{-\beta E_i} \end{align}\] which is called the partition function. Putting this together, the probability is \[\begin{align} P_i &= \frac{e^{-\beta E_i}}{Z} \\ &= \frac{\textit{Boltzmann factor}}{\textit{partition function}} \end{align}\] At this point, we haven't yet solved for \(\beta\), and to do so, we'd need to invoke the internal energy constraint: \[\begin{align} U &= \sum_i E_i P_i \\ U &= \frac{\sum_i E_i e^{-\beta E_i}}{Z} \end{align}\] As it turns out, \(\beta =\frac{1}{k_BT}\). This follows from my claim that the Lagrange multiplier is the partial derivative with respect to the constaint value \[\begin{align} k_B\beta &= \left(\frac{\partial S}{\partial U}\right)_{\text{Normalization=1}} \end{align}\]

However, I did not prove this to you. I will leave demonstrating this as a homework problem.

Homework for week 1 (PDF)

  1. Energy, Entropy, and Probabilities The internal energy and entropy can each be defined as a weighted average over microstates: \[\begin{align} U &= \sum_i E_i P_i & S &= -k_B\sum_i P_i \ln P_i \end{align}\] We also saw that the probability of each microstate could be defined in terms of a Lagrange multiplier \(\beta\) as \[\begin{align} P_i &= \frac{e^{-\beta E_i}}{Z} & Z &= \sum_i e^{-\beta E_i} \end{align}\] We can put these probabilities into the above weighted averages in order to relate \(U\) and \(S\) to \(\beta\). Do so, and see what connections you can make with thermodynamics that you know. In particular, the thermodynamic identity gives a relationship between internal energy and entropy. \[\begin{align} dU = TdS - pdV \end{align}\]
  2. Gibbs entropy is extensive Consider two noninteracting systems \(A\) and \(B\). We can either treat these systems as separate, or as a single combined system \(AB\). We can enumerate all states of the combined by enumerating all states of each separate system. The probability of the combined state \((i_A,j_B)\) is given by \(P_{ij}^{AB} = P_i^AP_j^B\). In other words, the probabilities combine in the same way as two dice rolls would, or the probabilities of any other uncorrelated events.

    1. Show that the entropy of the combined system \(S_{AB}\) is the sum of entropies of the two separate systems considered individually, i.e. \(S_{AB} = S_A+S_B\). THis means that entropy is extensive. Use the Gibbs entropy for this computation. You need make no approximation in solving this problem.

    2. Show that if you have \(N\) identical non-interacting systems, their total entropy is \(NS_1\) where \(S_1\) is the entropy of a single system.

    In real materials, we treat properties as being extensive even when there are interactions in the system. In this case, extensivity is a property of large systems, in which surface effects may be neglected.
  3. Boltzmann probabilities Consider the three-state system with energies \((-\epsilon,0,\epsilon)\) that we discussed in class.

  1. At infinite temperature, what are the probabilities of the three states being occupied?

  2. At very low temperature, what are the three probabilities?

  3. What happens to the probabilities if you allow the temperature to be negative?

Week 2: Entropy and Temperature

This week we will be following Chapter 2 of Kittel and Kroemer, which uses a microcanonical approach (or Boltzmann entropy approach) to relate entropy to temperature. This is an alternative derivation to the Gibbs approach we used last week, and it can be helpful to have seen both. In a few ways the Boltzmann approach is conceptually simpler, while there are a number of other ways in which the Gibbs approach is simpler.

Fundamental assumption

The difference between these two approaches is in what is considered the fundamental assumption. In the Gibbs entropy approach we assumed that the entropy was a "nice" function of the probabilities of microstates, which gave us the Gibbs formula. From there, we could maximize the entropy to find the probabilities under some set of constraints.

The Boltzmann approach makes what is perhaps simpler assumption, which is that if only microstates with a given energy are permitted, then all of the microstates with that energy are equally probable. (This scenario with all microstates having the same energy is the microcanonical ensemble.) Thus the macrostate with the most corresponding microstates will be most probable macrostate. The number of microstates corresponding to a given macrostate is called the multiplicity \(g(E,V)\). In this approach, multiplicity (which did not show up last week!) becomes a fundamentally important quantity, since the macrostate with the highest multiplicity is the most probable macrostate.

Outline of the week

One or two topics per day:

  1. Quick version showing the conclusions we will reach.
  2. Finding the multiplicity of a paramagnet (Chapter 1).
  3. Combining two non-interacting systems; defining temperature.
  4. Central limit theorem and how "large \(N\)" does its magic.

Quick version

This quick version will tell you all the essential physics results for the week, without proof. The beauty of statistical mechanics (whether following the text or using the information-theory approach of last week) is that you don't actually need to take on either faith or experiment the connection between the statistical theory and the empirical definitions used in thermodynamics.


The multiplicity sounds sort of like entropy (since it is maximized), but the multiplicity is not extensive (nor intensive), because the number of microstates for two identical systems taken together is the square of the number of microstates available to one of the single systems. This naturally leads to the Boltzmann definition of the entropy, which is \[\begin{align} S(E,V) = k_B\ln g(E,V). \end{align}\]

The logarithm converts the multiplicity into an extensive quantity, in a way that is directly analogous to the logarithm that appears in the Gibbs entropy.

For large systems (e.g. systems composed of \(\sim 10^{23}\) particles$), the most probable configuration is essentially the same as any remotely probable configuration. This comes about due for the same reason that if you flip \(10^{23}\) coins, you will get \(5\times 10^{22} \pm 10^{12}\) heads. On an absolute scale, that's a lot of uncertainty in the number of heads that would show up, but on a fractional scale, you're pretty accurate if you assume that 50% of the flips will be heads.


From Energy and Entropy (and last week), you will remember that \(dU = TdS - pdV\), which tells us that \(T = \left(\frac{\partial U}{\partial S}\right)_V\). If we assume that only states with one particular energy \(E\) have a non-zero probability of being occupied, then \(U=E\), i.e. the thermodynamic internal energy is the same as the energy of any allowed microstate. Then we can replace \(U\) with \(E\) and conclude that \[\begin{align} T &= \left(\frac{\partial E}{\partial S}\right)_V \\ \frac1T &= \left(\frac{\partial S}{\partial E}\right)_V \\ &= \left(\frac{\partial k_B\ln g(E,V)}{\partial E}\right)_V \\ &= k_B \frac1g \left(\frac{\partial g}{\partial E}\right)_V \end{align}\]

From this perspective, it looks like our job is to learn to solve for \(g(E)\) and from that to find \(S(E)\), and once we have done those tasks we will know the temperature (and soon everything else).

Differentiable multiplicity

The above assumes that \(g(E)\) is a differentiable function, which means that the number of microstates must be a continuous function of energy! This highlights one of the distinctions between the microcanonical approach and our previous (cannonical) Gibbs approach.

In reality, we know from quantum mechanics that any system of finite size has a finite number of eigenstates within any given energy range, and thus \(g(E)\) cannot be either continuous or differentiable. Boltzmann, of course, did not know this, and assumed that there were an infinite number of microstates possible within any energy range, and would strictly speaking interpret \(g(E)\) in terms of a volume of phase space.

The resolution to this conundrum is to invoke large numbers, and to assume that we are averaging \(g(E)\) over a range of energies in which there are many, many states. For real materials with \(N\approx 10^{23}\), this assumption is pretty valid. Much of this chapter will involve learning to work with this large \(N\) assumption, and to use it to extract physically meaningful results. In the Gibbs approach this large \(N\) assumption was not needed.

As Kittel discusses towards the end of the chapter, we only really need to know \(g(E)\) up to some constant factor, since a constant factor in \(g\) becomes a constant additive change in \(S\), which doesn't have any physical impact.

The "real" \(g(E)\) is a smoothed average over a range of energies. In practice, doing this can be confusing, and so we tend to focus on systems where the energy is always an integer multiple of some constant. Thus a focus on spins in a magnetic field, and harmonic oscillators.

Multiplicity of a paramagnet

So now the question becomes how to find the number of microstates that correspond to a given energy \(g(E)\). Once we have this in an analytically tractable form, we can everything else we might care for (with effort). This is essentially a counting problem, and much of what you need is introduced in Chapter 1. We will spend some class time going over one example of computing the multiplicity. Consider a paramagnetic system consisting of spin \(\frac12\) particles that can be either up or down. Each spin has a magnetic moment in the \(\hat z\) direction of \(\pm m\), and we are interested in the total magnetic moment \(M\), which is the sum of all the individual magnetic moments.

Small Group Question

Work out how many ways a system of 4 spins can have any possible magnetization of enumerating all the microstates corresponding to each magnetization.

Now find a mathematical expression that will tell you the multiplicity of a system with an even number \(N\) spins and just one \(\uparrow\) spin. Then find the multiplicity for two \(\uparrow\) spins, and for three \(\uparrow\) spins.

Now find a mathematical expression that will tell you the multiplicity of a system with an even number \(N\) spins and magnetization \(M=2sm\) where \(s\) is an integer. We call \(s\) the spin excess, since \(N_\uparrow = \frac12N + s\). Alternatively, you could write your expression in terms of the number of up spins \(N_\uparrow\) and the number of down spins \(N_\downarrow\).

We can enumerate all spin microstates:
\(\downarrow\downarrow\downarrow\downarrow\) g=1
\(\downarrow\downarrow\downarrow\uparrow\) \(\downarrow\downarrow\uparrow\downarrow\) \(\downarrow\uparrow\downarrow\downarrow\) \(\uparrow\downarrow\downarrow\downarrow\) g=4
\(\downarrow\downarrow\uparrow\uparrow\) \(\downarrow\uparrow\uparrow\downarrow\) \(\uparrow\uparrow\downarrow\downarrow\) \(\uparrow\downarrow\downarrow\uparrow\) \(\uparrow\downarrow\uparrow\downarrow\) \(\downarrow\uparrow\downarrow\uparrow\) g=6
\(\uparrow\uparrow\uparrow\downarrow\) \(\uparrow\uparrow\downarrow\uparrow\) \(\uparrow\downarrow\uparrow\uparrow\) \(\downarrow\uparrow\uparrow\uparrow\) g=4
\(\uparrow\uparrow\uparrow\uparrow\) g=1
To generalize this to \(g(N,s)\), we need to come up with a systematic way to count the states that have the same spin excess \(s\). Clearly if \(s=\pm N/2\), \(g=1\), since that means that all the spins are pointed the same way, and there is only one way to do that. \[\begin{align} g(N,s=\pm \frac12N) &= 1 \end{align}\] Now if we have just one spin going the other way, there are going to be \(N\) ways we could manage that: \[\begin{align} g\left(N,s=\pm \left(\frac12N-1\right)\right) &= N \end{align}\] Now when we go to flip it so we have two spins up, there will be \(N-1\) ways to flip the second spin. But then, when we do this we will end up counting every possibility twice, which means that we will need to divide by two. \[\begin{align} g\left(N,s=\pm \left(\frac12N-2\right)\right) &= N(N-1)/2 \end{align}\] When we get to adding the third \(\uparrow\) spin, we'll have \(N-2\) spins to flip. But now we have to be even more careful, since for the same three up-spins, we have several ways to reach that microstate. In fact, we will need to divide by \(6\), or \(3\times 2\) to get the correct answer (as we can check for our four-spin example). \[\begin{align} g\left(N,s=\pm \left(\frac12N-3\right)\right) &= \frac{N(N-1)(N-2)}{3!} \end{align}\] At this stage we can start to see the pattern, which comes out to \[\begin{align} g\left(N,s\right) &= \frac{N!}{\left(\frac12 N + s\right)!\left(\frac12N -s\right)!} \\ &= \frac{N!}{N_\uparrow!N_\downarrow!} \end{align}\]

Stirling's approximation

As you can see, we now have a bunch of factorials. Once we compute the entropy, we will have a bunch of logarithms of factorials. \[\begin{align} N! &= \prod_{i=1}^N i \\ \ln N! &= \ln\left(\prod_{i=1}^N i\right) \\ &= \sum_{i=1}^N \ln i \end{align}\] So you can see that the log of a factorial is a sum of logs. When the number of things being summed is large, we can approximate this sum with an integral. This may feel like a funny business, particularly for those of you who took my computational class, where we frequently used sums to approximate integrals! But the approximation can go both ways. In this case, if we approximate the integral as a sum we can find an analytic expression for the factorial: \[\begin{align} \ln N! &= \sum_{i=1}^N \ln i \\ &\approx \int_1^{N} \ln x dx \\ &= \left.x \ln x - x\right|_{1}^{N} \\ &= N\ln N - N + 1 \end{align}\] At this point, we should recognize that the \(1\) that we see is much smaller than the other two terms, and is actually likely to be wrong. Importantly, there is a larger error being made here, which we can see if we zoom into the upper end of our integral. We are missing \(\frac12 \ln N\)! The reason is that our integral went precisely to \(N\), but if we imagine a midpoint rule picture (or trapezoidal rule) we are missing half of that last point. This gives us: \[\begin{align} \ln N! &\approx \left(N+\frac12\right)\ln N - N \end{align}\]

We could find the constant term correctly (it is not 1), but that is more work, and even the \(\frac12\) above is usually omitted when using Stirling's approximation, since it is much smaller than the others when \(N\gg 0\)

Entropy of our spins

I'm going to use a different approach than the text to find the entropy of this spin system when there are many spins and the spin excess is relatively small. \[\begin{align} S &= k\ln g\left(N,s\right) \\ &= k\ln\left(\frac{N!}{\left(\tfrac12 N + s\right)!\left(\tfrac12N -s\right)!}\right) \\ &= k\ln\left(\frac{N!}{N_\uparrow!N_\downarrow!}\right) \\ &= k\ln\left(\frac{N!}{\left(h + s\right)!\left(h -s\right)!}\right) \end{align}\] At this point I'm going to define for convenience \(h\equiv \tfrac12 N\), just to avoid writing so many \(\tfrac12\). I'm also going to focus on the \(s\) dependence of the entropy. \[\begin{align} \frac{S}{k} &= \ln\left(N!\right) - \ln\left(N_\uparrow!\right) - \ln\left(N_\downarrow!\right) \\ &= \ln N! - \ln(h+s)! - \ln(h-s)! \\ &= \ln N! - \sum_{i=1}^{h+s} \ln i - \sum_{i=1}^{h-s} \ln i \end{align}\] At the last step, I wrote the log of the factorial as a sum of logs. Now we can pull out the bit that is independent of \(s\) to separate the \(s=0\) value from the contribution due to \(s\). \[\begin{align} \frac{S}{k} &= \ln N! -2\sum_{i=1}^h \ln i - \sum_{i=h+1}^{h+s} \ln i + \sum_{i=h-s+1}^{h} \ln i \\ &= \ln N! - 2\ln h - \sum_{i=h+1}^{h+s} \ln i + \sum_{i=h-s+1}^{h} \ln i \\ &= \frac{S_0}{k} - \sum_{i=h+1}^{h+s} \ln i + \sum_{i=h-s+1}^{h} \ln i \end{align}\] In the last step here, I defined \(S_0\) to be the entropy when \(s=0\) for convenience. We now have a couple of sums to simplify. \[\begin{align} \frac{S-S_0}{k} &= -\sum_{i=h+1}^{h+s} \ln i + \sum_{i=h-s+1}^{h} \ln i \\ &= -\sum_{k=1}^{s} \ln(h+ k) + \sum_{k=1}^{s} \ln (h-k+1) \\ &= \sum_{k=1}^{s} \left(\ln (h-k+1) - \ln(h+ k)\right) \\ &= \sum_{k=1}^{s} \left(\ln h + \ln\left(1-\frac{k-1}{h}\right) - \ln h - \ln\left(1+ \frac{k}{h}\right)\right) \\ &= \sum_{k=1}^{s} \left(\ln\left(1-\frac{k-1}{h}\right) - \ln\left(1+ \frac{k}{h}\right)\right) \end{align}\] Now we make our first approximation: we assume \(s\ll N\), which means that \(s\ll h\). \[\begin{align} \frac{S-S_0}{k} &\approx \sum_{k=1}^{s} \left(-\frac{k-1}{h} - \frac{k}{h}\right) \\ &= -\frac2{h}\sum_{k=1}^{s} \left(k-\tfrac12\right) \end{align}\]
Sum to integral
Sum to integral
Now we have this sum to solve. You can find this sum either geometrically or with calculus. The calculus involves turning the sum into an integral. As you can see in the figure, the integral \[\begin{align} \int_0^s x dx = \tfrac12 s^2 \end{align}\]

has the same value as the sum.

Sum geometrically
Sum geometrically

The geometric way to solve this looks visually very much the same as the integral picture, but instead of computing the area from the straight line, we cut the stair-step area "half" and fit the two pieces together such that they form a rectangle with width \(s/2\) and height \(s\).

Taken together, this tells us that when \(s\ll N\) \[\begin{align} S(N,s) &\approx S(N,s=0) - k\frac{2}{h}\frac{s^2}{2} \\ &= S(N,s=0) - k\frac{2s^2}{N} \end{align}\] This means that the multiplicity is gaussian: \[\begin{align} S &= k \ln g \\ g(N,s) &= e^{\frac{S(N,s)}{k}} \\ &= e^{\frac{S(N,s=0)}{k} - \frac{2s^2}{N}} \\ &= g(N,s=0)e^{-\frac{2s^2}{N}} \end{align}\]

Thus the multiplicity (and thus probability) is peaked at \(s=0\) as a gaussian with width \(\sim\sqrt{N}\). This tells us that the width of the peak increases as we increase \(N\). However, the excess spin per particle decreases as \(\sim\frac{1}{\sqrt{N}}\). So that means that our fractional polarization becomes far more sharply peaked as we increase \(N\).

Thermal contact

Suppose we put two systems in contact with one another. This means that energy can flow from one system to the other. We assume, however, that the contact between the two systems is weak enough that their energy eigenstates are unaffected. This is a bit of a contradiction you'll need to get used to: we treat our systems as non-interacting, but assume there is some energy transfer between them. The reasoning is that the interaction between them is very small, so that we can treat each system separately, but energy can still flow.

We ask the question: "How much energy will each system end up with after we wait for things to settle down?" The answer to this question is that energy will settle down in the way that maximizes the number of microstates.

For the moment, let's consider two systems in contact:

System \(A\)
A system of 10 spins each with energy \(\pm \frac12\). This system has the following multiplicity found from Pascal's triangle: \[\begin{equation*}\small \begin{array}{ccccccccccc} 1 \\ 1 & 1 \\ 1 & 2 & 1 \\ 1 & 3 & 3 & 1 \\ 1 & 4 & 6 & 4 & 1 \\ 1 & 5 & 10 & 10 & 5 & 1 \\ 1 & 6 & 15 & 20 & 15 & 6 & 1 \\ 1 & 7 & 21 & 35 & 35 & 21 & 7 & 1 \\ 1 & 8 & 28 & 56 & 70 & 56 & 28 & 8 & 1 \\ 1 & 9 & 36 & 84 & 126 & 126 & 84 & 36 & 9 & 1 \\ 1 & 10 & 45 & 120 & 210 & 252 & 210 & 120 & 45 & 10 & 1 \\ \hline -5 & -4 & -3 & -2 & -1 & 0 & 1 & 2 & 3 & 4 & 5 \end{array} \end{equation*}\]
System \(B\)
Three objects that can each have energy 1-6, as in three 6-sided dice. This system has the following multiplicity: \[\begin{equation*}\small \begin{array}{cr|rrrrrrrrrrrrrrrrrrrrrrrrr} E & g \\ \hline 3 & 1 & 1\times 111 \\ 4 & 3 & 3\times 211 \\ 5 & 6 & 3\times 311 & 3\times 221 \\ 6 & 10 & 3\times 411 & 6\times 321 & 1\times 222 \\ 7 & 15 & 3\times 511&6\times 421&3\times 331&3\times 322 \\ 8 & 21 & 3\times 611 & 6\times 521 & 6\times 431 & 3\times 422 & 3\times 322 \\ 9 & 25 & 6\times 621 & 6\times 531 & 3\times 522 & 3\times 441 & 6\times 432 & 1\times 333 \\ 10 & 27 & 6\times 653 & 3\times 644 & 6\times 541 & 6\times 532 & 3\times 442 & 3\times 433 \\ 11 & 27 & 6\times 641 & 6\times 632 & 3\times 551 & 6\times 542 & 3\times 533 & 3\times 443 \\ 12 & 25 & 6\times 651 & 6\times 642 & 3\times 633 & 3\times 552 & 6\times 543 & 1\times 444 \\ 13 & 21 & 3\times 661 & 6\times 652 & 6\times 643 & 3\times 553 & 3\times 544 \\ 14 & 15 & 3\times 662 & 6\times 653 & 3\times 644 & 3\times 554 \\ 15 & 10 & 3\times 663 & 6\times 654 & 1\times 555 \\ 16 & 6 & 3\times 664 & 3\times 655 \\ 17 & 3 & 3\times 665 \\ 18 & +\hfill 1 & 1\times 666 \\\hline 6^3 = & 216 \end{array} \end{equation*}\]

We now want to consider these two systems taken together. You might wonder why I made these examples so very large. The answer is that we are going to be looking at \(\frac{dg}{dE}\), and we need something large enough that the slope is actually interesting. Also, these two systems demonstrate the universal property that when you combine many similar systems together you get a multiplicity that is increasingly sharply peaked.

Given the above, we know for each possible energy of systems \(A\) or \(B\) how many microstates there are. The total number of microstates for system \(A\) is \(2^{10} = 1024\), while the total number of microstates for system \(B\) is \(6^3=216\).

What is the total number of microstates when you consider these systems as a single combined system?
We need to multiply the numbers of microstates, because for each microstate of \(A\), it could be in any microstate of \(B\). So the total is \(2^{10}3^6=221184\). Let's not attempt to enumerate all these microstates.

Since we have two separate systems here, it is meaningful to ask what the probability is system \(A\) to have a given energy, given that the combined system has energy \(E_{AB}\).

Small group question
Work out the probability for system \(A\) to have every possible energy (please end up with a decimal number), given that the total energy \(E_{AB}\) has a fixed value that I will tell you. What is the most probable energy for system \(B\)? Total energies: 1,2,3,4,5,6,7,8.
Probability distribution when E_{AB}=1
Probability distribution when \(E_{AB}=1\)
Probability distribution when E_{AB}=5
Probability distribution when \(E_{AB}=5\)
Probability distribution when E_{AB}=8
Probability distribution when \(E_{AB}=8\)
\[\begin{equation*} \begin{array}{cc|cl} E_B & E_A & P_B & \\\hline 3 & -2 & 0.37 & 1\times 120 \\ 4 & -3 & 0.42 & 3\times 45 \\ 5 & -4 & 0.18 & 6\times 10 \\ 6 & -5 & 0.03 & 10\times 1 \\ \end{array} \end{equation*}\] \[\begin{equation*} \begin{array}{cc|cl} E_B & E_A & P_B & \\\hline 3 & 2 & 0.02 & 1\times 120 \\ 4 & 1 & 0.09 & 3\times 210 \\ 5 & 0 & 0.20 & 6\times 252 \\ 6 & -1 & 0.28 & 10\times 210 \\ 7 & -2 & 0.24 & 15\times 120 \\ 8 & -3 & 0.13 & 21\times 45 \\ 9 & -4 & 0.03 & 25\times 10 \\ 10 & -5 & 0.00 & 27\times 1 \\ \end{array} \end{equation*}\] \[\begin{equation*} \begin{array}{cc|cl} E_B & E_A & P_B & \\\hline 3 & 4 & 0.00 & 1\times 10 \\ 4 & 3 & 0.01 & 3\times 45 \\ 5 & 2 & 0.05 & 6\times 120 \\ 6 & 1 & 0.13 & 10\times 210 \\ 7 & 0 & 0.24 & 15\times 252 \\ 8 & -1 & 0.28 & 21\times 210 \\ 9 & -2 & 0.19 & 25\times 120 \\ 10 & -3 & 0.08 & 27\times 45 \\ 11 & -4 & 0.02 & 27\times 10 \\ 12 & -5 & 0.00 & 25\times 1 \\ \end{array} \end{equation*}\] \[\begin{equation*} \begin{array}{cc|cl} E_B & E_A & P_B & \\\hline 3 & 5 & 0.00 & 1\times 1 \\ 4 & 4 & 0.00 & 3\times 10 \\ 5 & 3 & 0.01 & 6\times 45 \\ 6 & 2 & 0.06 & 10\times 120 \\ 7 & 1 & 0.16 & 15\times 210 \\ 8 & 0 & 0.27 & 21\times 252 \\ 9 & -1 & 0.26 & 25\times 210 \\ 10 & -2 & 0.16 & 27\times 120 \\ 11 & -3 & 0.06 & 27\times 45 \\ 12 & -4 & 0.01 & 25\times 10 \\ 13 & -5 & 0.00 & 21\times 1 \\ \end{array} \end{equation*}\]

Why did I make you do all this work?

  1. This probability distribution has a peak. As we go to larger numbers, this peak will get sharper. As you adjusted the total energy, the peak shifted to a different \(E_B\).

  2. There is a pattern here that tells us what the temperature is! (Or what it is related to...)

Given that these two systems are able to exchange energy, they ought to have the same temperature. What is the same between the two systems when we are at the most probable energy splitting? Let's look at the properties. As an example, let's pick \(E_{AB}=7\), for which \(E_B=8\) and \(E_A=-1\) was the most probable.
\[\begin{equation*} \begin{array}{cc|cc|cc|ccc} E_B & E_A & g_A & g_B & g_A' & g_B' & g_A'/g_A & g_B'/g_B \\\hline -1^{-} & 6^+ & 252 & 21 & \sim 66 & \sim 4.5 & \sim .31 & \sim .45 \\ \end{array} \end{equation*}\]

The closest here are the ratios of the slope of the multiplicity to the multiplicity itself. With this few particles, the discreteness makes taking derivatives challenging, so it's not a terribly convincing demonstration, but we can also prove that this ratio should be equal on the two sides, provided the multiplicity is smooth.

The probability of a given energy \(E_B\) given that the total energy is \(E_{AB}\) is given by \[\begin{align} P(E_B|E_{AB}) = \frac{g_B(E_B)g_A(E_{AB}-E_B)}{ \sum_{E} g_B(E)g_A(E_{AB}-E) } \end{align}\] To find the energy with maximum probability, we can take a derivative with respect to \(E_B\) and set it to zero: \[\begin{align} \frac{dP}{dE_B} &= 0 \\ &= \frac{g_B'(E_B)g_A(E_{AB}-E_B) - g_B(E_B)g_A'(E_{AB}-E_B)}{ \sum_{E} g_B(E)g_A(E_{AB}-E) } \end{align}\] We note that the denominator can be multiplied out, simplifying things: \[\begin{align} g_B'(E_B)g_A(E_{AB}-E_B) &= g_B(E_B)g_A'(E_{AB}-E_B) \\ \frac{g_B'(E_B)}{g_B(E_B)} &= \frac{g_A'(E_{AB}-E_B)}{g_A(E_{AB}-E_B)} \end{align}\] So indeed, our inference that the ratio of this slope to value is equal on the two sides was correct. How do we turn this into a temperature? One way is to recognize this pattern of drivative of a function over the function as the derivative of a logarithm. \[\begin{align} \frac{d \ln g}{dE} &= \frac{\frac{dg}{dE}}{g(E)} \end{align}\]

Then we can note that this logarithm is an extensive quantity, which smells like an entropy, given that its derivative seems to relate to temperature. Making this relationship explicit requires an actual definition of temperature (rather than simply the fact that the temperature of two systems in thermal equilibrium are equal.

As you already know, \[\begin{align} T = \left(\frac{\partial U}{\partial S}\right)_{V} \end{align}\] so it should be unsurprising (plus you already know) that \[\begin{align} S &= k_B\ln g \\ \frac1{T} &= \left(\frac{\partial S}{\partial E}\right)_{V} \end{align}\]

Recap from 1st homework

In your solution to your first homework problem from last week, you showed that \[\begin{align} S &= k\beta U + k\ln Z \\ U &= TS - kT \ln Z \end{align}\] We can now turn this around and identify \[\begin{align} - kT \ln Z &= U - TS \end{align}\]

as the Helmholtz free energy \(F\equiv U-TS\), which you already learned about in Energy and Entropy. To the extent that you have a physical interpretation for the free energy, this gives you a physical interpretation for the partition function, beyond simply being a normalization constant.

Homework for week 2 (PDF)

  1. Entropy and Temperature (K&K 2.1) Suppose \(g(U) = CU^{3N/2}\), where \(C\) is a constant and \(N\) is the number of particles.
    1. Show that \(U=\frac32 N k_BT\).
    2. Show that \(\left(\frac{\partial^2S}{\partial U^2}\right)_N\) is negative. This form of \(g(U)\) actually applies to the ideal gas.
  2. Paramagnetism (K&K 2.2) Find the equilibrium value at temperature \(T\) of the fractional magnetization \[\begin{equation} \frac{M}{Nm} \equiv \frac{2\langle s\rangle}{N} \end{equation}\] of the system of \(N\) spins each of magnetic moment \(m\) in a magnetic field \(B\). The spin excess is \(2s\). The energy of this system is given by \[\begin{align} U &= -MB \end{align}\] Take the entropy as the logarithm of the multiplicity \(g(N,s)\) as given in (1.35 in the text): \[\begin{equation} S(s) \approx k_B\log g(N,0) - k_B\frac{2s^2}{N} \end{equation}\] for \(|s|\ll N\). Hint: Show that in this approximation \[\begin{equation} S(U) = S_0 - k_B\frac{U^2}{2m^2B^2N}, \end{equation}\]

    with \(S_0=k_B\log g(N,0)\). Further, show that \(\frac1{kT} = -\frac{U}{m^2B^2N}\), where \(U\) denotes \(\langle U\rangle\), the thermal average energy.

  3. Quantum harmonic oscillator (K&K 2.3)
    1. Find the entropy of a set of \(N\) oscillators of frequency \(\omega\) as a function of the total quantum number \(n\). Use the multiplicity function (1.55): \[\begin{equation} g(N,n) = \frac{(N+n-1)!}{n!(N-1)!} \end{equation}\] and assume that \(N\gg 1\). This means you can make the Sitrling approximation that \(\log N! \approx N\log N - N\). It also means that \(N-1 \approx N\).
    2. Let \(U\) denote the total energy \(n\hbar\omega\) of the oscillators. Express the entropy as \(S(U,N)\). Show that the total energy at temperature \(T\) is \[\begin{equation} U = \frac{N\hbar\omega}{e^{\frac{\hbar\omega}{kT}}-1} \end{equation}\]

      This is the Planck result: it is derived again in Chapter 4 by a powerful method that does not require us to find the multiplicity function.

      David notes
      In this chapter we are doing things the hard way. The "powerful" method coming up is actually the easy way, which is how you will always solve problems like this, once you have been taught it!

Week 3: Boltzmann distribution and Helmholtz

This week we will be deriving the Boltzmann ratio and the Helmholtz free energy, continuing the microcanonical approach we started last week. Last week we saw that when two systems were considered together in a microcanonical picture, the energy of each system taken individually is not fixed. This provides our stepping stone to go from a microcanonical picture where all possible microstates have the same energy (and equal probability) to a canonical picture where all energies are possible and the probability of a given microstate being observed depends on the energy of that state.

We ended last week by defining temperature by \[\begin{align} \frac{1}{T} &= \left(\frac{\partial S}{\partial E}\right)_{V} \end{align}\]

where \(S(E)\) is the Boltzmann expression for the entropy of a microcanonical ensemble. To definitively connect this with temperature, we will again consider two systems in thermal equilibrium using a microcanonical ensemble, but this time we will make one of those two systems huge. In fact, it will be so huge that we can treat it using classical thermodynamics, i.e. we can conclude that the above equation applies, and we can assume that the temperature of this huge system is unaffected by the small change in energy that could happen due to differences in the small system.

Let us now examine the multiplicity of our combined system, making \(B\) be our large system: \[\begin{align} g_{AB}(E) &= \sum_{E_A}g_A(E_A) g_B(E_{AB}-E_A) \end{align}\] We can further find the probability of any particular energy being observed from \[\begin{align} P_{A}(E_A) &= \frac{g_A(E_A)g_B(E_{AB}-E_A)}{ \sum_{E_A'}g_A(E_A') g_B(E_{AB}-E_A') } \end{align}\] where we are counting how many microstatesstates of the combined system have this particular energy in system \(A\), and dividing by the total number of microstates of the combined system to create a probability. So far this is identical to what we had last week. The difference is that we are now claiming that system \(B\) is huge. This means that we can treat \(g_B\) in a simple approximate manner. \[\begin{align} g_B(E_{AB}-E_A) &= e^{S_B(E_{AB}-E_A)/k_B} \\ &\approx e^{S_B(E_{AB})/k_B - \frac{E_A}{k_BT} + \cdots} \end{align}\] In the approximate step here, I used a power series expansion of the entropy around \(E_{AB}\), and also used the fact the the inverse temperature is the derivative of the entropy with respect to energy. Now provided the system \(B\) is very seriously huge, we can omit the \(\cdots\) and consider the equation exact. Plugging this into the probability equation we have \[\begin{align} P_{A}(E_A) &= \frac{g_A(E_A) e^{S_B(E_{AB})/k_B - \frac{E_A}{k_BT}}}{ \sum_{E_A'}g_A(E_A') e^{S_B(E_{AB})/k_B - \frac{E_A'}{k_BT}} } \\ &= \frac{g_A(E_A) e^{- \frac{E_A}{k_BT}}}{ \sum_{E_A'}g_A(E_A') e^{- \frac{E_A'}{k_BT}} } \end{align}\] Now this looks a bit different than the probabilities we saw previously (two weeks ago), and that is because this is the probability that we see energy \(E_A\), not the probability for a given microstate, and thus it has the factors of \(g_A\), and it sums over energies rather than microstates. We can easily rectify this, which will result in \[\begin{align} P_\mu^A &= \frac{e^{- \frac{E_\mu^A}{k_BT}}}{Z} \\ Z &= \sum_\mu e^{- \frac{E_\mu^A}{k_BT}} \end{align}\]

This is all there is to show the Boltzmann probability distribution from the microcanonical picture: Big system with little system, treat big system thermodynamically, count microstates.

Internal energy

Now that we have the set of probabilities, there are a few things we can solve for directly, namely any quantities that are directly defined from probabilities. Most specifically, the internal energy \[\begin{align} U &= \sum_\mu P_\mu E_\mu \\ &= \sum_\mu E_\mu \frac{e^{-\beta E_\mu}}{Z} \\ &= \frac1Z \sum_\mu E_\mu e^{-\beta E_\mu} \end{align}\] Now doing yet another summation will often feel tedious. There are a couple of ways to make this easier. The simplest is to examine the sum above and notice how very similar it is to the partition function itself. If you take a derivative of the partition function with respect to \(\beta\), you will find: \[\begin{align} \left(\frac{\partial Z}{\partial \beta}\right)_{V} &= \sum_\mu e^{-\beta E_\mu} (-E_\mu) \\ &= -UZ \\ U &= -\frac{1}{Z}\left(\frac{\partial Z}{\partial \beta}\right)_{V} \\ &= -\left(\frac{\partial \ln Z}{\partial \beta}\right)_{V} \end{align}\]
Big Warning
In this class, I do not want you beginning any solution (either homework or exam) with a formula for \(U\) in terms of \(Z\)! This step is not that hard, and you need to do it every time. What you need to remember is definitions, which in this case is how \(U\) comes from probability. The reasoning here is that I've all too often seen students who years after taking thermal physics can only remember that there is some expression for \(U\) in terms of \(Z\). It is easier and more correct to remember that \(U\) is a weighted average of the energy.


How do we compute pressure? So far, everything we have done has kept the volume fixed. Pressure tells us how the energy changes when we change the volume, i.e. how much work is done. From Energy and Entropy, we know that \[\begin{align} dU &= TdS - pdV \\ p &= -\left(\frac{\partial U}{\partial V}\right)_S \end{align}\]

So how do we find the pressure? We need to find the change in internal energy when we change the volume at fixed entropy.

Small white boards
How do we keep the entropy fixed when changing the volume?
Experimentally, we would avoid allowing any heating by insulating the system. Theoretically, this is less easy. When we consider the Gibbs entropy, if we could keep all the probabilities fixed while expanding, we would also fix the entropy! In quantum mechanics, we can show that such a process is possible using time-dependent perturbation theory. Under certain conditions, if you perturb a system sufficiently slowly, it will remain in the "same" eigenstate it was in originally. Although the eignestate changes, and its energy changes, they do so continuously.
If we take a derivative of \(U\) with respect to volume while holding the probabilities fixed, we obtain the following result: \[\begin{align} p &= -\left(\frac{\partial U}{\partial V}\right)_S \\ &= -\left(\frac{\partial \sum_\mu E_\mu P_\mu}{\partial V}\right)_S \\ &= -\sum_\mu P_\mu \frac{d E_\mu}{d V} \end{align}\]

Helmholtz free energy

We saw a hint above that \(U\) somehow relates to \(\ln Z\), which hinted that \(\ln Z\) might be something special. Let's put this into thermodynamics language.1 \[\begin{align} U &= -\left(\frac{\partial \ln Z}{\partial \beta}\right)_{V} \\ d\ln Z &= -Ud\beta + \left(\frac{\partial \ln Z}{\partial V}\right)_{\beta} dV \end{align}\] Now we're going to try a switch to a \(dU\) rather than a \(d\beta\), since we know something about \(dU\). \[\begin{align} d(\beta U) &= Ud\beta + \beta dU \\ d\ln Z &= -\left(d(\beta U) - \beta dU\right) + \left(\frac{\partial \ln Z}{\partial V}\right)_{\beta} dV \\ &= \beta dU -d(\beta U) + \left(\frac{\partial \ln Z}{\partial V}\right)_{\beta} dV \\ \beta dU &= d\left(\ln Z + \beta U\right) - \left(\frac{\partial \ln Z}{\partial V}\right)_{\beta} dV \\ dU &= Td\left(k_B\ln Z + U/T\right) - kT\left(\frac{\partial \ln Z}{\partial V}\right)_{T} dV \end{align}\] Comparing this result with the thermodynamic identity tells us that \[\begin{align} S &= k_B\ln Z + U/T \\ F &\equiv U-TS \\ &= U - T\left(k_B\ln Z + U/T\right) \\ &= U - k_BT\ln Z + U \\ &= - k_BT\ln Z \end{align}\]

That was a bit of a differentials slog, but got us the same result without assuming the Gibbs entropy. It did, however, demonstrate a not-quite-contradiction, in that the expression we found for the entropy is not mathematically equal to the Boltmann entropy. It approaches the same thing for large systems, although I won't prove that now.

Small groups
Consider a system with \(g\) eigenstates, each with energy \(E_0\). What is the free energy?
We begin by writing down the partition function \[\begin{align} Z &= \sum_\mu e^{-\beta E_\mu} \\ &= g e^{-\beta E_0} \end{align}\] Now we just need a log and we're done. \[\begin{align} F &= -kT\ln Z \\ &= -kT \ln\left(g e^{-\beta E_0}\right) \\ &= -kT \left(\ln g + \ln e^{-\beta E_0}\right) \\ &= E_0 - Tk\ln g \end{align}\]

This is just what we would have concluded about the free energy if we had used the Boltzmann expression for the entropy in this microcanonical ensemble.

Waving our hands, we can understand \(F=-kT\ln Z\) in two ways:

  1. If there are more accessible microstates, \(Z\) is bigger, which means \(S\) is bigger and \(F\) must be more negative.
  2. If we only consider the most probable energies, to find the energy from \(Z\), we need the negative logarithm, and a \(kT\) to cancel out the \(\beta\).

Using the free energy

Why the big deal with the free energy? One way to put it is that it is relatively easy to compute. The other is that once you have an analytic expression for the free energy, you can solve for pretty much anything else you want.

Recall: \[\begin{align} F &\equiv U-TS \\ dF &= dU - SdT - TdS \\ &= -SdT - pdV \\ -S &= \left(\frac{\partial F}{\partial T}\right)_V \\ -p &= \left(\frac{\partial F}{\partial V}\right)_T \end{align}\]

Thus by taking partial derivatives of \(F\) we can find \(S\) and \(p\) as well as \(U\) with a little arithmetic. You have all seen the Helmholtz free energy before so this shouldn't be much of a surprise. Practically, the Helmholtz free energy is why finding an analytic expression for the partition function is so valuable.

In addition to the "fundamental" physical parameters, we can also find response functions, such as heat capacity or compressibility which are their derivatives. Of particular interest is the heat capacity at fixed volume. The heat capacity is vaguely defined as: \[\begin{align} C_V &\equiv \sim \left(\frac{\bar d Q}{\partial T}\right)_V \end{align}\] by which I mean the amount of heat required to change the temperature by a small amount, divided by that small amount, while holding the volume fixed. The First Law tells us that the heat is equal to the change in internal energy, provided no work is done (i.e. holding volume fixed), so \[\begin{align} C_V &= \left(\frac{\partial U}{\partial T}\right)_V \end{align}\] which is a nice equation, but can be a nuisance because we often don't know \(U\) as a function of \(T\), which is not one of its natural variables. We can also go back to our Energy and Entropy relationship between heat and entropy where \(\bar dQ = TdS\), and use that to find the ratio that defines the heat capacity: \[\begin{align} C_V &= T \left(\frac{\partial S}{\partial T}\right)_V. \end{align}\]

Note that this could also have come from a manipulation of the previous derivative of the internal energy. However, the "heat" reasoning allows us to recognize that the heat capacity at constant pressure will have the same form when expressed as an entropy derivative. This expression is also convenient when we compute the entropy from the Helmholtz free energy, because we already know the entropy as a function of \(T\).

Ideal gas with just one atom

Let us work on the free energy of a particle in a 3D box.

Small groups
Work out (or write down) the energy eigenstates for a particle confined to a cubical volume with side length \(L\). You may either use periodic boundary conditions or an infinite square well. When you have done so, write down an expression for the partition function.
The energy is just the kinetic energy, given by \[\begin{align} T &= \frac{\hbar^2 |\vec k|^2}{2m} \end{align}\] The allowed values of \(k\) are determined by the boundary conditions. If we choose periodic boundary conditions, then \[\begin{align} k_x &= n_x \frac{2\pi}{L} & n_x &= \text{any integer} \end{align}\] and similarly for \(k_y\) and \(k_z\), which gives us \[\begin{align} E_{n_xn_yn_z} &= \frac{2\pi^2\hbar^2}{mL^2}\left(n_x^2+n_y^2+n_z^2\right) \end{align}\] where \(n_x\), \(n_y\), and \(n_z\) take any integer values. If we chose the infinite square well boundary conditions instead, our integers would be positive values only, and the prefactor would differ by a factor of four.
From this point, we just need to sum over all states to find \(Z\), and from that the free energy and everything else! So how do we sum all these things up? \[\begin{align} Z &= \sum_{n_x=-\infty}^{\infty} \sum_{n_y=-\infty}^{\infty} \sum_{n_z=-\infty}^{\infty} e^{-\beta \frac{2\pi^2\hbar^2}{mL^2}\left(n_x^2+n_y^2+n_z^2\right) } \\ &= \sum_{n_x}\sum_{n_y} \sum_{n_z} e^{-\beta \frac{2\pi^2\hbar^2}{mL^2}n_x^2 } e^{-\beta \frac{2\pi^2\hbar^2}{mL^2}n_y^2 } e^{-\beta \frac{2\pi^2\hbar^2}{mL^2}n_z^2 } \\ &= \left(\sum_{n_x} e^{-\beta \frac{2\pi^2\hbar^2}{mL^2}n_x^2 }\right) \left(\sum_{n_y} e^{-\beta \frac{2\pi^2\hbar^2}{mL^2}n_y^2 }\right) \left(\sum_{n_z} e^{-\beta \frac{2\pi^2\hbar^2}{mL^2}n_z^2 }\right) \\ &= \left(\sum_{n=-\infty}^\infty e^{-\beta \frac{2\pi^2\hbar^2}{mL^2}n^2 }\right)^3 \end{align}\]

The last bit here basically looks a lot like separation of variables. Our energy separates into a sum of \(x\), \(y\) and \(z\) portions (which is why we can use separation of variables for the quantum problem), but that also causes things to separate (into a product) when we compute the partition function.

This final sum here is now something we would like to approximate. If our box is reasonably big (and our temperature is not too low), we can assume that \(\frac{4\pi^2\hbar^2}{k_BTmL^2} \ll 1\), which is the classical limit. In this limit, the "thing in the exponential" hardly changes when we change \(n\) by 1, so we can reasonably replace this summation with an integral.

You might have thought to use a power series expansion (which is a good instinct!) but in this case that won't work, because \(n\) gets arbitrarily large.
\[\begin{align} Z &\approx \left(\int_{-\infty}^\infty e^{-\frac{2\pi^2\hbar^2}{k_BTmL^2}n^2 } dn\right)^3 \end{align}\] We can now do a \(u\) substitution to simplify this integral. \[\begin{align} \xi &= \sqrt{\frac{2\pi^2\hbar^2}{k_BTmL^2}}n & d\xi &= \sqrt{\frac{2\pi^2\hbar^2}{k_BTmL^2}}dn \end{align}\] This gives us a very easy integral. \[\begin{align} Z &= \left(\sqrt{\frac{k_BTmL^2}{2\pi^2\hbar^2}}\int_{-\infty}^\infty e^{-\xi^2 } d\xi\right)^3 \\ &= \left(\frac{k_BTmL^2}{2\pi^2\hbar^2}\right)^\frac32 \left(\int_{-\infty}^\infty e^{-\xi^2 } d\xi\right)^3 \\ &= V\left(\frac{k_BTm}{2\pi^2\hbar^2}\right)^\frac32\pi^\frac32 \\ &= V\left(\frac{k_BTm}{2\pi\hbar^2}\right)^\frac32 \end{align}\] So there we have our partition function for a single atom in a big box. Let's go on to find exciting things! First off, let's give a name to the nasty fraction to the \(\frac32\) power. It has dimensions of inverse volume, or number per volume, and it has \(\hbar\) in it (which makes it quantum) so let's call it \(n_Q\), since I use \(n=N/V\) for number density. \[\begin{align} n_Q &= \left(\frac{k_BTm}{2\pi\hbar^2}\right)^\frac32 \\ F &= -kT\ln Z \\ &= -kT\ln \left(Vn_Q\right) \end{align}\]

This looks like (and is) a very simple formula, but you need to keep in mind that \(n_Q\) depends on temperature, so it's not quite as simple as it looks. Now that we have the Helmholtz free energy, we can solve for the entropy, pressure, and internal energy pretty quickly.

Small groups
Solve for the entropy, pressure, and internal energy. \[\begin{align} S &= -\left(\frac{\partial F}{\partial T}\right)_V \\ &= k\ln \left(Vn_Q\right) +\frac{kT}{Vn_Q}V\frac{dn_Q}{dT} \\ &= k\ln \left(Vn_Q\right) +\frac{kT}{n_Q}\frac32 \frac{n_Q}{T} \\ &= k\ln \left(Vn_Q\right) +\frac32k_B \end{align}\] You could find \(U\) by going back to the weighted average definition and using the derivative trick from the partition function, but with the free energy and entropy it is just algebra. \[\begin{align} U &= F + TS \\ &= -kT\ln \left(Vn_Q\right) + kT\ln \left(Vn_Q\right) +\frac32k_BT \\ &= \frac32k_BT \end{align}\] The pressure derivative gives a particularly simple result. \[\begin{align} p &= -\left(\frac{\partial F}{\partial V}\right)_T \\ &= -\frac{kT}{V} \end{align}\]

Ideal gas with multiple atoms

Extending from a single atom to several requires just a bit more subtlety. Naively, you could just argue that because we understand extensive and intensive quantities, we should be able to go from a single atom to \(N\) atoms by simply scaling all extensive quantities. That is almost completely correct (if done correctly).

The entropy has an extra term (the "entropy of mixing"), which also shows up in the free energy. Note that while we may think of this "extra term" as an abstract counting thing, it is physically observable, provided we do the right kind of experiment (which turns out to need to involve changing \(N\), so we won't discuss it in detail until we talk about changing \(N\) later).

There are a few different ways we could imagine putting \(N\) non-interacting atoms together. I will discuss a few here, starting from simplest, and moving up to the most complex.

Different atoms, same box
One option is to consider a single box with volume \(V\) that holds \(N\) different atoms, each of a different kind, but all with the same mass. In this case, each microstate of the system will consist of a microstate for each atom. Quantum mechanically, the wave function for the entire state with \(N\) atoms will separate and will be a product of states \[\begin{align} \Psi_\mu(\vec r_1, \vec r_2, \cdots, \vec r_N) &= \prod_{i=1}^{N}\phi_{n_{xi},n_{yi},n_{zi}}(\vec r_i) \end{align}\] and the energy will just be a sum of different energies. The result of this will be that the partition function of the entire system will just be the product of the partition functions of all the separate non-interacting systems (which happen to all be equal). This is mathematically equivalent to what already happened with the three \(x\), \(y\) and \(z\) portions of the partition function. \[\begin{align} Z_N &= Z_1^N \\ F_N &= N F_1 \end{align}\] This results in simply scaling all of our extensive quantities by \(N\) except the volume, which didn't increase when we added more atoms.
Identical atoms, but different boxes
We can also consider saying all atoms are truly identical, but each atom is confined into a different box, each with identical (presumably small) size. In this case, the same reasoning as we used above applies, but now we also scale the total volume up by \(N\). This is a more natural application of the idea of extensivity. \[\begin{align} Z_N &= Z_1^N \\ F_N &= N F_1 \\ V &= N V_1 \end{align}\] This is an example of taking the idea of extensivity to an extreme: we keep saying that a system with half as much volume and half as many atoms is "half as much" until there is only one atom left. You would be right to be skeptical that putting one atom per box hasn't resulted is an error.
Identical atoms, same box

This is the picture for a real ideal gas. All of our atoms are the same, or perhaps some fraction are a different isotope, but who cares about that? Since they are all in the same box, we will want to write the many-atom wavefunction as a product of single-atom wavefunctions (sometimes called orbitals). However, now we have a bit of a challenge. Since there is no difference between the different atoms, there is no difference between atom 1 having the state \((n_x,n_y,n_z) = 1,2,3\) while atom 2 has the state \((n_x,n_y,n_z) = 3,2,1\) versus atom 2 having the first state, while atom 1 has the second state.

This is a little complicated to explain, and it'll be simpler if we just label each single-particle eigenstate (or orbital) with a single label for simplicity. If we think about \(N\) atoms, we can define an eigenstate according to how many atoms are in state 0, how many are in state 1, etc. This is a combination kind of problem, which we have encountered before. It simplifies if we assume that the odds of having two atoms in exactly the same state is negligible, in which case we will find that we are counting each microstate \(N!\) times. \[\begin{align} Z_N &= \frac1{N!}Z_1^N \\ F_N &= N F_1 - k_BT \ln N! \\ &\approx N F_1 - Nk_BT(\ln N - 1) \end{align}\]

We could go on, and the text does so, but we've got a whole chapter on the ideal gas, so I'll leave this for you to play with.

Homework for week 3 (PDF)

For each problem, please let me know how long the problem took, and what resources you used to solve it!

  1. Free energy of a two state system (K&K 3.1, modified)
    1. Find an expression for the free energy as a function of \(T\) of a system with two states, one at energy 0 and one at energy \(\varepsilon\).
    2. From the free energy, find expressions for the internal energy \(U\) and entropy \(S\) of the system.
    3. Plot the entropy versus \(T\). Explain its asymptotic behavior as the temperature becomes high.
    4. Plot the \(S(T)\) versus \(U(T)\). Explain the maximum value of the energy \(U\).
  2. Magnetic susceptibility (K&K 3.2)
    1. Use the partition function to find an exact expression for the magentization \(M\) and the susceptibility \(\chi\equiv\left(\frac{\partial M}{\partial B}\right)_T\) as a function of temperature and magnetic field for the model system of magnetic moments in a magnetic field. The result for the magnetization is \(M=nm\tanh\left(\frac{mB}{kT}\right)\), as derived in (46) by another method. Here \(n\) is the particle concentration. The result is plotted in the book.
    2. Find the free energy and epress the result as a function only of \(T\) and the parameter \(x\equiv \frac{M}{nm}\).
    3. Show that the susceptibility is \(\chi=\frac{nm^2}{kT}\) in the limit \(mB\ll kT\).
  3. Free energy of a harmonic oscillator (K&K 3.3) A one-dimensional harmonic oscillator has an infinite series of equally spaced energy states, with \(\varepsilon_s = s\hbar\omega\), where \(s\) is a positive integer or zero, and \(\omega\) is the classical frequency of the oscillator. We have chosen the zero of energy at the state \(s=0\) [which Kittel can get away with here, but is not actually the zero of energy!].
    1. Show that for a harmonic oscillator the free energy is \[\begin{equation} F = k_BT\log\left(1 - e^{-\frac{\hbar\omega}{k_BT}}\right) \end{equation}\] Note that at high temperatures such that \(k_BT\gg\hbar\omega\) we may expand the argument of the logarithm to obtain \(F\approx k_BT\log\left(\frac{\hbar\omega}{kT}\right)\).
    2. From the free energy above, show that the entropy is \[\begin{equation} \frac{S}{k_B} = \frac{\frac{\hbar\omega}{kT}}{e^{\frac{\hbar\omega}{kT}}-1} - \log\left(1-e^{-\frac{\hbar\omega}{kT}}\right) \end{equation}\]

      This entropy is shown in the nearby figure, as well as the heat capacity.

      Entropy of a simple harmonic oscillator
      Entropy of a simple harmonic oscillator
      Heat capacity of a simple harmonic oscillator
      Heat capacity of a simple harmonic oscillator
  4. Energy fluctuations (K&K 3.4) Consider a system of fixed volume in thermal contact with a resevoir. Show that the mean square fluctuations in the energy of the system is \[\begin{equation} \left<(\varepsilon-\langle\varepsilon^2\rangle\right> = k_BT^2\left(\frac{\partial U}{\partial T}\right)_{V} \end{equation}\] Here \(U\) is the conventional symbol for \(\langle\varepsilon\rangle\). Hint: Use the partition function \(Z\) to relate \(\left(\frac{\partial U}{\partial T}\right)_V\) to the mean square fluctuation. Also, multiply out the term \((\cdots)^2\). Note: The temperature \(T\) of a system is a quantity that by definition does not fluctuate in value when the system is in thermal contact with a resevoir. Any other attitude would be inconsistent with our definition of the temperature of a system. The energy of such a system may fluctuate, but the temperature does not. Some workers do not adhere to a rigorous defintion of temperature. Thus Landau and Lifshitz give the result \[\begin{align} \left<(\Delta T)^2\right> = \frac{k_BT^2}{C_V}, \end{align}\]

    but this should be viewed as just another form of what you have proven in this problem, with \(\Delta T\) set equal to \(\Delta U/C_V\). We know that \(\Delta U = C_V\Delta T\) (for small values of \(\Delta T\)), whence these two equations become equal.

  5. Quantum concentration (K&K 3.8) Consider one particle confined to a cube of side \(L\); the concentration in effect is \(n=L^{-3}\). Find the kinetic energy of the particle when in the ground state. There will be a value of the concentration for which this zero-point quantum kinetic energy is equal to the temperature \(kT\). (At this concentration the occupancy of the lowest orbital is of the order of unity; the lowest orbital always has a higher occupancy than any other orbital.) Show that the concentration \(n_0\) thus defined is equal to the quantum concentration \(n_Q\) defined by (63): \[\begin{equation} n_Q \equiv \left(\frac{MkT}{2\pi\hbar^2}\right)^{\frac32} \end{equation}\]

    within a factor of the order of unity.

  6. One-dimensional gas (K&K 3.11) Consider an ideal gas of \(N\) particles, each of mass \(M\), confined to a one-dimensional line of length \(L\). Find the entropy at temperature \(T\). The particles have spin zero.

Week 4: Thermal radiation and Planck distribution

This week we will be tackling things that reduce to a bunch of simple harmonic oscillators. Any system that classically reduces to a set of normal modes each with its own frequency falls in this category. We will start with just an ordinary simple harmonic oscillator, and will move on to look at radiation (photons) and sound in solids (phonons).

Harmonic oscillator

You will recall that the energy eigenvalues of a single simple harmonic oscillator are given by \[\begin{align} E_n &= (n+\tfrac12)\hbar\omega \end{align}\]
The text uses \(s\) rather than \(n\) for the quantum number, but that is annoying, and on the blackboard my \(s\) looks too much like my \(S\), so I'll stick with \(n\). The text also omits the zero-point energy. It does make the math simpler, but I think it's worth seeing how the zero-point energy goes away to start with.
We will begin by solving for the properties of a single simple harmonic oscillator at a given temperature. You already did this once using multiplicities, but it's easier this way. \[\begin{align} Z &= \sum_{n=0}^{\infty} e^{-(n+\tfrac12)\beta\hbar\omega} \\ &= e^{-\tfrac12\beta\hbar\omega}\sum_{n=0}^{\infty} e^{-n\beta\hbar\omega} \end{align}\] Now the sum is actually a harmonic (or geometric) sum, which has a little trick to solve: \[\begin{align} Z &= e^{-\tfrac12\beta\hbar\omega}\sum_{n=0}^{\infty} \left(e^{-\beta\hbar\omega}\right)^n \\ \xi &\equiv e^{-\beta\hbar\omega} \end{align}\] The trick involves multiplying the series by \(\xi\) and subtracting: \[\begin{align} \Xi &= \sum_{n=0}^{\infty} \xi^n \\ &= 1 + \xi + \xi^2 + \cdots \\ \xi\Xi &= \xi + \xi^2 + \cdots \\ \Xi - \xi\Xi &= 1 \\ \Xi &= \frac1{1-\xi} \end{align}\] Thus we find that the partition function is simply \[\begin{align} Z &= \frac{e^{-\tfrac12\beta\hbar\omega}}{1 - e^{-\beta\hbar\omega}} \end{align}\] This gives us the free energy \[\begin{align} F &= -kT\ln Z \\ &= -kT\ln\left(\frac{e^{-\tfrac12\beta\hbar\omega}}{1 - e^{-\beta\hbar\omega}}\right) \\ &= \frac12\hbar\omega + kT\ln\left(1 - e^{-\beta\hbar\omega}\right) \end{align}\]

We can see now that the ground state energy just ends up as a constant that we add to the free energy, which is what you probably would have guessed. Kittel was able to omit this constant simply by redefining the zero of energy.

Small groups
Solve for the high temperature limit of the free energy.
At high temperatures, \(\beta\hbar\omega \ll 1\), which means \[\begin{align} e^{-\beta\hbar\omega} &= 1 - \beta\hbar\omega + \frac12(\beta\hbar\omega)^2+\cdots \\ \ln(1-e^{-\beta\hbar\omega}) &= \ln\left(\beta\hbar\omega - \tfrac12(\beta\hbar\omega)^2+\cdots\right) \\ F &\approx \tfrac12\hbar\omega + kT\ln\left(\frac{\hbar\omega}{kT}\right) \end{align}\] So far this doesn't tell us much, but from it we can quickly tell the high temperature limits of the entropy and internal energy: \[\begin{align} S &\approx -k\ln\left(\frac{\hbar\omega}{kT}\right) - kT\frac{kT}{\hbar\omega}\left(-\frac{\hbar\omega}{kT^2}\right) \\ &= k\left(\ln\left(\frac{kT}{\hbar\omega}\right)+1\right) \end{align}\] The entropy increases as we increase temperature, as it always must. The manner in which \(S\) increases logarithmically with temperature tells us that the number of accessible microstates must be proportional to \(\frac{kT}{\hbar\omega}\).
\[\begin{align} S &= -\left(\frac{\partial F}{\partial T}\right)_V \\ &= -k\ln\left(1 - e^{-\beta\hbar\omega}\right) + kT\frac{e^{-\beta\hbar\omega}}{1 - e^{-\beta\hbar\omega}}\frac{\hbar\omega}{kT^2} \\ &= -k\ln\left(1 - e^{-\beta\hbar\omega}\right) + \frac{e^{-\beta\hbar\omega}}{1 - e^{-\beta\hbar\omega}}\frac{\hbar\omega}{T} \end{align}\]

Planck distribution

Finally, we can find the internal energy and the average quantum number (or number of "phonons"). The latter is known as the Planck distribution. \[\begin{align} U &= F + TS \\ &= \frac12\hbar\omega + T \frac{e^{-\beta\hbar\omega}}{1 - e^{-\beta\hbar\omega}}\frac{\hbar\omega}{T} \\ &= \left(\frac12 + \frac{e^{-\beta\hbar\omega}}{1 - e^{-\beta\hbar\omega}}\right)\hbar\omega \\ U &= \left(\langle n\rangle + \tfrac12\right)\hbar\omega \\ \langle n\rangle &= \frac{e^{-\beta\hbar\omega}}{1 - e^{-\beta\hbar\omega}} \\ &= \frac{1}{e^{\beta\hbar\omega}-1} \end{align}\]

So far, all we've done is a straightforward application of the canonical formalism from last week: we computed a partition function, took a log, and from that found the entropy and internal energy.

Small groups
Solve for the high-temperature and low-temperature limits of the internal energy and/or the average number of quanta \(\langle n\rangle\).
High temperature answer
First we consider \(\frac{kT}{\hbar\omega}\gg 1\) or \(\beta\hbar\omega\ll 1\). In this case, the exponential is going to be very close to one, and we can use a power series approximation for it. \[\begin{align} \langle n\rangle &= \frac{1}{e^{\beta\hbar\omega}-1} \\ &\approx \frac{1}{\left(1 + \beta\hbar\omega + \tfrac12(\beta\hbar\omega)^2 +\cdots\right) -1} \\ &= \frac{kT}{\hbar\omega} \frac1{1 + \tfrac12\beta\hbar\omega + \cdots} \\ &= \frac{kT}{\hbar\omega} \left(1 - \tfrac12\beta\hbar\omega + \cdots\right) \\ &\approx \frac{kT}{\hbar\omega} - \frac12 \end{align}\] The first term is our equipartition term: \(\frac12kT\) each for the kinetic and potential energy. The second term is our next-order correction, which you need not necessarily include. There would be a next term which would be proportional to \(1/T\), but we have omitted.
Low temperature answer
At low temperature \(\beta\hbar\omega\gg 1\), and we would rather look the other representation: \[\begin{align} \langle n\rangle &= \frac{1}{e^{\beta\hbar\omega}-1} \\ &= \frac{e^{-\beta\hbar\omega}}{1 - e^{-\beta\hbar\omega}} \end{align}\] because now the exponentials are small (rather than large), which means we can expand the denominator as a power series. \[\begin{align} \langle n\rangle &= e^{-\beta\hbar\omega}\left(1 + e^{-\beta\hbar\omega} + \cdots\right) \\ &\approx e^{-\beta\hbar\omega} + e^{-2\beta\hbar\omega} \end{align}\] Once again, I kept one more term than is absolutely needed. Clearly at low temperature we have a very low number of quanta, which should be no shock. I hope you all expected that the system would be in the ground state at very low temperature.

Summing over microstates

I realized that we haven't spent much time talking about how to sum over microstates. Once you "get it," summing over microstates is very easy. Unfortunately, this makes it less obvious that this requires teaching, and I have a tendency to skim over this summation.

A nice example of this was the second homework, which involved the paramagnet again. You needed to find the partition function for \(N\) dipoles. After spending a week working with multiplicities, it would be very natural to take the \[\begin{align} Z \equiv \sum_\mu e^{-\beta E_\mu} \end{align}\] and think of the \(\mu\) as having something to do with spin excess, and to think that this sum should involve multiplicities. You can write a solution here using multiplicities and summing over all possible energies, but that is the hard way. The easy way only looks easy once you know how to do it. The easy way involves literally summing over every possible sequence of spins. \[\begin{align} Z &= \sum_{s_1=\pm1}\sum_{s_2=\pm1}\cdots\sum_{s_N=\pm1}e^{-\beta E(s_1,s_2,\cdots,s_N} \end{align}\] This may look messy, but things simplify when we consider the actual energy (unless we try to simplify that by expressing it in terms of \(N_\uparrow\) or the spin excess). \[\begin{align} E(s_1,s_2,\cdots,s_N) &= -s_1mB - s_2mB - \cdots - s_NmB \end{align}\] Now this may look pretty nasty, but it's actually beautiful, because each \(s_i\) has a separate term that is added together, which means that it separates! I'll use fewer words for a bit... \[\begin{align} Z &= \sum_{s_1=\pm1}\sum_{s_2=\pm1}\cdots\sum_{s_N=\pm1} e^{\beta (s_1mB + s_2mB + \cdots + s_NmB)} \\ &= \sum_{s_1=\pm1}\sum_{s_2=\pm1}\cdots\sum_{s_N=\pm1} e^{\beta s_1mB}e^{\beta s_2mB}\cdots \\ &= \sum_{s_1=\pm1}e^{\beta s_1mB}\sum_{s_2=\pm1}e^{\beta s_2mB}\cdots \sum_{s_N=\pm1}e^{\beta s_NmB} \\ &= \left(\sum_{s_1=\pm1}e^{\beta s_1mB}\right)\cdots \left(\sum_{s_N=\pm1}e^{\beta s_NmB}\right) \\ &= \left(\sum_{s=\pm1}e^{\beta s mB}\right)\cdots \left(\sum_{s=\pm1}e^{\beta smB}\right) \\ &= \left(\sum_{s=\pm1}e^{\beta s mB}\right)^N \end{align}\]

The important steps above were

  1. Writing the sum over states as a nested sum over every quantum number of the system.
  2. Breaking the exponential into a product, which we can do because the energy is a sum of terms each of which involve just one quantum number.
  3. Doing each sum separately, and finding the result as the product of all those sums.

Note that the final result here is a free energy that is just \(N\) times the free energy for a system consisting of a single spin. And thus we could alternatively do our computation for a system with a single spin, and then multiply everything that is extensive by \(N\). The latter is a valid shortcut, but you should know why it gives a correct answer, and when (as when we have identical particles) you could run into trouble.

Black body radiation

Researchers in 1858-1860 realized that materials emitted light in strict proportion to their ability to absorb it, and hence a perfectly black body would be emit the most radiation when heated. Planck and others realized that we should be able to use thermal physics to predict the spectrum of black body radation. A key idea was to recognize that the light itself should be in thermal equilibrium.

One example of a "black body" is a small hole in a large box. Any light that goes into the hole will bounce around so many times before coming out of the hole, that it will all be absorbed. This leads to the idea of studying the radiation in a closed box, which should match that of a black body, when it is in thermal equilibrium.

Eigenstates of an empty box

So what are the properties of an empty box? Let's assume metal walls, and not worry too much about details of the boundary conditions, which shouldn't make much difference provided the box is pretty big. The reasoning is basically the same as for a particle in a 3D box: the waves must fit in the box. As for the particle in a box, we can choose either periodic boundary conditions or put nodes at the boundaries. I generally prefer periodic (which gives both positive and negative \(\vec k\)), rather than dealing with sine waves (which are superpositions of the above). A beautiful thing about periodic boundary conditions is that your set of \(\vec k\) vectors is independent of the Hamiltonian, so this looks very much like the single atom in a box we did last week. \[\begin{align} k_x &= n_x \frac{2\pi}{L} & n_x &= \text{any integer} \end{align}\] and similarly for \(k_y\) and \(k_z\), which gives us \[\begin{align} \omega(\vec k) &= c|\vec k| \\ \omega_{n_xn_yn_z} &= c\sqrt{n_x^2+n_y^2+n_z^2} \end{align}\] where now we need to be extra-careful to rememer that \(n_x\) is not a number of photons, even though \(n\) is a number of photons. Fortunately, we will soon be done with our \(n_x\), once we finish summing. Now the trick here is that for each mode there is a sum over all the quantum numbers, and we need to use a product over all modes. This is going to look pretty heinous. Note also that for every \(\vec k\), there are two polarizations of photon, so we need to square it all. \[\begin{align} Z &= \left(\prod_{n_x=-\infty}^{\infty} \prod_{n_y=-\infty}^{\infty} \prod_{n_z} \sum_{n=0}^{\infty} e^{-\frac{\beta h c}{L}\sqrt{n_x^2+n_y^2+n_z^2}(n+\frac12) } \right)^2 \end{align}\] Fortunately, the inmost sum over \(n\) we have already done. But this time, let's leave out the zero point energy by dropping the \(\frac12\) from the above. Thus we get \[\begin{align} Z &= \left( \prod_{n_x=-\infty}^{\infty} \prod_{n_y=-\infty}^{\infty} \prod_{n_z=-\infty}^{\infty} \frac{1}{1 - e^{-\frac{\beta h c}{L}\sqrt{n_x^2+n_y^2+n_z^2}}} \right)^2. \end{align}\] This still probably doesn't leave you dancing with joy. Fortunately, there are quite a few simplifications left. Before we start simplifying further, let's go ahead and look at the Helmholtz free energy. \[\begin{align} F &= -kT\ln Z \\ &= -2kT\ln\left( \prod_{n_x=-\infty}^{\infty} \prod_{n_y} \prod_{n_z} \frac{1}{1 - e^{-\frac{\beta h c}{L}\sqrt{n_x^2+n_y^2+n_z^2}}} \right) \\ &= 2kT \sum_{n_x=-\infty}^{\infty} \sum_{n_y} \sum_{n_z} \ln\left( 1 - e^{-\frac{\beta h c}{L}\sqrt{n_x^2+n_y^2+n_z^2}} \right) \end{align}\] This is starting to look more friendly. As usual, we'd like to take a large-volume approximation, which will mean that \(\frac{\beta hc}{L}\ll 1\). In this limit (as usual), we can turn our summation into an integration. \[\begin{align} F \approx 2kT \iiint_{-\infty}^\infty \ln\left( 1 - e^{-\frac{\beta h c}{L}\sqrt{n_x^2+n_y^2+n_z^2}} \right)dn_x dn_y dn_z \end{align}\] This integral is begging to be done in spherical coordinates, and I'll just define \(n\equiv \sqrt{n_x^2+n_y^2+n_z^2}\) for that integral. \[\begin{align} F &\approx 2kT \int_{0}^\infty \ln\left( 1 - e^{-\frac{\beta h c}{L}n}\right)4\pi n^2dn \\ &= 8\pi kT\left(\frac{LkT}{hc}\right)^3 \int_0^\infty \ln\left(1-e^{-\xi}\right)\xi^2d\xi \\ &= 8\pi \frac{V(kT)^4}{h^3c^3} \int_0^\infty \ln\left(1-e^{-\xi}\right)\xi^2d\xi \end{align}\]

At this point we have pretty much solved for the free energy of a vacuum at temperature \(T\). We know it should scale as \(T^4\), and that it should scale with volume. The latter should have been obvious, since it's the only way that the free energy could be extensive. You might worry about the definite integral, but it is just a dimensionless number! Yes, it matters if we want to know precisely how much radiation to expect from a black body, but we could solve this integral numerically, or we could have just done an experimental measurement to see what this number is. Wolfram alpha tells us that this number is -2.165. You could tell it should be negative because the thing in the \(\ln\) is always less than one, meaning the \(\ln\) is always negative. You might be weirded out that the free energy density is negative, but this just means it is dominated by entropy, since the entropy term is always negative.

Extra fun
Can any of you find an elegant solution to the above definite integral? If so, please share it with me, and I'll share it with the class.
We can solve for the entropy straight off: \[\begin{align} S &= -\left(\frac{\partial F}{\partial T}\right)_V \\ &= -32\pi k V \left(\frac{kT}{hc}\right)^3 \int_0^\infty \ln\left(1-e^{-\xi}\right)\xi^2d\xi \end{align}\] and as usual, we'll want to know the internal energy \[\begin{align} U &= F+TS \\ &= (8-32)\pi V \frac{(kT)^4}{h^3c^3} \int_0^\infty \ln\left(1-e^{-\xi}\right)\xi^2d\xi \\ \frac{U}{V} &= -24\pi \frac{(kT)^4}{h^3c^3} \int_0^\infty \ln\left(1-e^{-\xi}\right)\xi^2d\xi \end{align}\]

So the internal energy scales the same as the free energy, but is positive, as we would expect.

Sefan-Boltzmann law of radiation

To find the radiation power, we need do a couple of things. One is to multiply the energy per volume by the speed of light, which would tell us the energy flux through a hole if all the energy were passing straight through that hole. However, there is an additional geometrical term we will need to find the actual magnitude of the power, since the radiation is travelling equally in all directions. This will give us another dimensionless factor.

If we define the velocity as \(c\hat k\) where \(c\) is the speed of light and \(\hat k\) is its direction, the power flux (or intensity) in the \(\hat z\) direction will be given by \[\begin{align} I &= \frac{U}{V} \frac12\left<\left|c \hat k\cdot \hat z\right|\right> \end{align}\] where the average is over all possible directions, and the \(\frac12\) comes about because half will be going in entirely the wrong direction. \[\begin{align} I &= \frac{U}{V}c\int_0^{\frac{\pi}{2}}\cos\theta\sin\theta d\theta \\ &= \frac{U}{V}c\int_0^{1}\xi d\xi \\ &= \frac{U}{V}\frac{c}{2} \\ &= 12\pi \frac{(kT)^4}{h^3c^2} \int_0^\infty \ln\left(1-e^{-\xi}\right)\xi^2d\xi \end{align}\]

This is the famous Stefan-Boltzmann law of radiation.

Side note
Why is this \(T^4\) law important for incandescent light bulbs? The resistivity of a metal increases with temperature. In a light bulb, if you have a bit of wire that is a bit hotter than the rest, its resistivity will be higher, and that will cause it to have more Joule heating, and get hotter. If nothing else came into play, we'd have a feedback loop, and the hot bit would just keep getting hotter, and having higher resistivity, until it vaporized. Boom. Fortunately, the power of light radiated from that hot bit of wire will increase faster than its resistivity goes up (because \(T^4\) is serious!), preventing the death spiral, and saving the day!

Planck radiation law

Having found the total power radiated, a fair question is how much of that power is at each possible frequency. This defines the black body spectrum. Each mode has an occupancy \(\langle n\rangle\) that is the same as that of the harmonic oscillator from Monday. But the power radiated also depends on how many modes there are at a given frequency.

Kirchoff law

Surface temperature

Phonons in solids

Debye theory

Homework for week 4 (PDF)

  1. Number of thermal photons (K&K 4.1) Show that the number of photons \(\sum\langle s_n\rangle\) in equilibrium at temperature \(T\) in a cavity of volume \(V\) is \[\begin{equation} N=2.404\pi^{-2}\left(\frac{kT}{\hbar c}\right)^3 \end{equation}\]

    From (23) the entropy is \(S=k_B\frac{4\pi V}{45}\left(\frac{kT}{\hbar c}\right)^3\), whence \(S/N \approx 3.602\). It is believed that the total number of photons in the universe is \(10^8\) times larger than the total number of nucleons (protons, neutrons). Because both entropies are of the order of the respective number of particles (see Eq. 3.76), the photons provide the dominant contribution to the entropy of the universe, although the particles dominate the total energy. We believe the entropy of the photons is essentially constant, so that the entropy of the universe is approximately constant with time.

    David notes: If the universe is governed by quantum mechanics, then its entropy is exactly constant with time. However, the entropy of the universe is not a measurable quantity (even in gendanken experiments), and not terribly useful to think about.

  2. Surface temperature of the sun (K&K 4.2) The value of the total radiant energy flux density at the Earth from the Sun normal to the incident rays is called the solar constant of the Earth. The observed value integrated over all emission wavelengths and referred to the mean Earth-Sun distance is: \[\begin{equation} \text{solar constant} = 0.136 \text{J}\text{s}^{-1}\text{cm}^{-2} \end{equation}\]
    1. Show that the total rate of energy generation of the Sun is \(4\times 10^{26} \text{Js}^{-1}\).
    2. From this result and the Sefan-Boltzmann constant \(\sigma_B=5.67\times10^{-12} \text{Js}^{-1}\text{cm}^{-2}\text{K}^{-4}\), show that the effective temperature of the surface of the Sun treated as a black body is \(T\approx 6000\text{K}\). Take the distance of the Earth from the Sun as \(1.5\times10^{13}\text{cm}\) and the radius of the Sun as \(7\times10^{10}\text{cm}\).
  3. Age of the sun (K&K 4.4) Suppose \(4\times 10^{26}\text{Js}^{-1}\) is the total rate at which the Sun radiates energy at the present time.
    1. Find the total energy of the Sun available for radiation, on the rough assumption that the energy source is the conversion of hydrogen (atomic weight 1.0078) to helium (atomic weight 4.0026) and that the reaction stops when 10 percent of the original hydrogen has been converted to helium. Use the Einstein relation \(E = (\Delta M)c^2\).
    2. Use (a) to estimate the expectancy of the Sun. It is believed that the age of the universe is about \(10\times 10^9\) years.
  4. Surface temperature of the earth (K&K 4.5) Calculate the temperature of the surface of the Earth on the assumption that as a black body in thermal equilibrium it reradiates as much thermal radiation as it receives from the Sun. Assume also that the surface of the Earth is a constant temperature over the day-night cycle. Use \(T_{\odot}=5800\text{K}\); \(R_{\odot}=7\times 10^{10}\text{cm}\); and the Earth-Sun distance of \(1.5\times 10^{13}\text{cm}\).

  5. Pressure of thermal radiation (K&K 4.6) Show for a photon gas that
    1. \[\begin{align} p&=\left(\frac{\partial U}{\partial V}\right)_S \\ &= -\sum_j s_j\hbar \left(\frac{d\omega_j}{dV}\right), \end{align}\] where \(s_j\) is the number of photons in the mode \(j\);
    2. \[\begin{align} \frac{d\omega_j}{dV} &= -\frac{\omega_j}{3V}; \end{align}\]
    3. \[\begin{align} p = \frac{U}{3V} \end{align}\]
  6. Heat shields (K&K 4.8) A black (nonreflective) plane at temperature \(T_u\) is parallel to a black plane at temperature \(T_l\). The net energy flux density in vacuum between the two planes is \(J_U=\sigma_B\left(T_u^4-T_l^4\right)\), where \(\sigma_B\) is the Stefan-Boltzmann constant used in (26). A third black plane is inserted between the other two and is allowed to come to a steady state temperature \(T_m\). Find \(T_m\) in terms of \(T_u\) and \(T_l\), and show that the net energy flux density is cut in half because of the presence of this plane. This is the principle of the heat shield and is widely used to reduce radiant heat transfer. Comment: The result for \(N\) independent heat shields floating in temperature between the planes \(T_u\) and \(T_l\) is that the net energy flux density is \(J_U=\sigma_B\frac{T_u^4-T_l^4}{N+1}\).

Week 5: Chemical potential and Gibbs distribution

Homework for week 5 (PDF)

  1. Centrifuge (K&K 5.1) A circular cylinder of radius \(R\) rotates about the long axis with angular velocity \(\omega\). The cylinder contains an ideal gas of atoms of mass \(M\) at temperature \(T\). Find an expression for the dependence of the concentration \(n(r)\) on the radial distance \(r\) from the axis, in terms of \(n(0)\) on the axis. Take \(\mu\) as for an ideal gas.

  2. Potential energy of gas in gravitational field (K&K 5.3) Consider a column of atoms each of mass \(M\) at temperature \(T\) in a uniform gravitational field \(g\). Find the thermal average potential energy per atom. The thermal average kinetic energy is independent of height. Find the total heat capacity per atom. The total heat capacity is the sum of contributions from the kinetic energy and from the potential energy. Take the zero of the gravitational energy at the bottom \(h=0\) of the column. Integrate from \(h=0\) to \(h=\infty\).

  3. Active transport (K&K 5.4) The concentration of potassium \(\text{K}^+\) ions in the internal sap of a plant cell (for example, a fresh water alga) may exceed by a factor of \(10^4\) the concentration of \(\text{K}^+\) ions in the pond water in which the cell is growing. The chemical potential of the \(\text{K}^+\) ions is higher in the sap because their concentration \(n\) is higher there. Estimate the difference in chemical potential at \(300\text{K}\) and show that it is equivalent to a voltage of \(0.24\text{V}\) across the cell wall. Take \(\mu\) as for an ideal gas. Because the values of the chemical potential are different, the ions in the cell and in the pond are not in diffusive equilibrium. The plant cell membrane is highly impermeable to the passive leakage of ions through it. Important questions in cell physics include these: How is the high concentration of ions built up within the cell? How is metabolic energy applied to energize the active ion transport?

    David adds
    You might wonder why it is even remotely plausible to consider the ions in solution as an ideal gas. The key idea here is that the ideal gas entropy incorporates the entropy due to position dependence, and thus due to concentration. Since concentration is what differs between the cell and the pond, the ideal gas entropy describes this pretty effectively. In contrast to the concentration dependence, the temperature-dependence of the ideal gas chemical potential will be almost entirely incorrect.
  4. Gibbs sum for a two level system (K&K 5.6)
    1. Consider a system that may be unoccupied with energy zero, or occupied by one particle in either of two states, one of energy zero and one of energy \(\varepsilon\). Show that the Gibbs sum for this system is \[\begin{align} \zeta = 1 + \lambda + \lambda e^{-\frac{\varepsilon}{kT}} \end{align}\] One assumption excludes the possibility of one particle in each state at the same time. Notice that we include in the sum a term for \(N=0\) as a particular state of a system of a variable number of particles.
    2. Show that the thermal average occupancy of the system is \[\begin{align} \langle N\rangle = \frac{\lambda + \lambda e^{-\frac{\varepsilon}{kT}}}{\zeta} \end{align}\]
    3. Show that the thermal average occupancy of the state at energy \(\varepsilon\) is \[\begin{align} \langle N(\varepsilon)\rangle = \frac{\lambda e^{-\frac{\varepsilon}{kT}}}{\zeta} \end{align}\]
    4. Find an expression for the thermal average energy of the system.
    5. Allow the possibility that the orbital at \(0\) and at \(\varepsilon\) may be occupied each by one particle at the same time; Show that \[\begin{align} \zeta &= 1 + \lambda + \lambda e^{-\frac{\varepsilon}{kT}} + \lambda^2 e^{-\frac{\varepsilon}{kT}} \\ &= (1+\lambda)\left(1+e^{-\frac{\varepsilon}{kT}}\right) \end{align}\] Because \(\zeta\) can be factored as shown, we have in effect two independent systems.
  5. Carbon monoxide poisoning (K&K 5.8) In carbon monoxide poisoning the CO replaces the O2 adsorbed on hemoglobin (\(\text{Hb}\)) molecules in the blood. To show the effect, consider a model for which each adsorption site on a heme may be vacant or may be occupied either with energy \(\varepsilon_A\) by one molecule O2 or with energy \(\varepsilon_B\) by one molecule CO. Let \(N\) fixed heme sites be in equilibrium with O2 and CO in the gas phases at concentrations such that the activities are \(\lambda(\text{O}_2) = 1\times 10^{-5}\) and \(\lambda(\text{CO}) = 1\times 10^{-7}\), all at body temperature \(37^\circ\text{C}\). Neglect any spin multiplicity factors.
    1. First consider the system in the absence of CO. Evaluate \(\varepsilon_A\) such that 90 percent of the \(\text{Hb}\) sites are occupied by O2. Express the answer in eV per O2.
    2. Now admit the CO under the specified conditions. Fine \(\varepsilon_B\) such that only 10% of the Hb sites are occupied by O2.

Week 6: Ideal gas

Homework for week 6 (PDF)

  1. 6.1

  2. 6.2

  3. 6.3

  4. 6.6

  5. 6.12

  6. 6.14

  7. 6.15

Week 7: Fermi and Bose gases

Homework for week 7 (PDF)

  1. 7.2

  2. 7.3

  3. 7.5

  4. 7.6

  5. 7.7

  6. 7.11

Week 8: Gibbs free energy and chemical reactions

Homework for week 8 (PDF)

  1. 8.1

  2. 8.2

  3. 8.3

  4. 8.5

  5. 8.6

  6. 8.7

Week 9: Phase transformations

Homework for week 9 (PDF)

  1. 9.1

  2. 9.2

  3. 9.3

Week 10: Buffer for running over? Binary mixtures?


Here are the solutions for all the homework problems. Although these solutions are available even before homework is due, I recommend that you do your best solve the homework problems without checking the solutions. I would encourage you to go so far as to not check the solutions until after you have turned in your homework. This will enable you to practice determining if your answer is correct without knowing what the correct answer is. This is what you will have to do after you have graduated (or on the exam)!

In any case, please include for each homework problem that you solve an explanation of what resources you made use of, whether it be help from other students, these solutions, or some other resource. Please explain in each case how you used the resource. e.g. did you look at the solutions to confirm that your answer was correct, or did you look at the solutions to see how to start the problem? Your grade on the homework will not be diminished, even if you essentially copied the solution.

I would also appreciate it if you let me know where you got stuck on a given problem. I may address your difficulty in class, or I may choose to change the homework problem for next year.

Please note that verbatim copying of any solution (whether it be these, or a homework from another student) is plagiarism, and is not permitted. If you wish to copy a solution, please use it as a guide for how to do the steps, but perform each step on your own, and ideally add some words of your own explaining what you are doing in each step.

Solution for week 1

PDF version of solutions

  1. Energy, Entropy, and Probabilities

    To begin, as the question prompts, we will stick the probabilities into our expressions for \(U\) and \(S\). If you knew how everything was going to work out, you could only stick them into the \(\ln P_i\), but I'll act as though I haven't solved this \(N\gg1\) times before. \[\begin{align} U &= \sum_i E_i \frac{e^{-\beta E_i}}{Z} \\ S &= -k_B\sum_i \frac{e^{-\beta E_i}}{Z} \ln\left(\frac{e^{-\beta E_i}}{Z}\right) \end{align}\] At this point, we can see that there is a chance to simplify the \(\ln\). \[\begin{align} S &= -k_B\sum_i \frac{e^{-\beta E_i}}{Z} \left(\ln\left(e^{-\beta E_i}\right) - \ln Z\right) \\ &= -k_B\sum_i \frac{e^{-\beta E_i}}{Z} \left(-\beta E_i - \ln Z\right) \\ &= k_B\sum_i \frac{e^{-\beta E_i}}{Z} \left(\beta E_i + \ln Z\right) \\ &= k_B\sum_i \frac{e^{-\beta E_i}}{Z}\beta E_i + k_B\sum_i \frac{e^{-\beta E_i}}{Z} \ln Z \\ &= k_B \beta \cancelto{U}{\sum_i \frac{e^{-\beta E_i}}{Z} E_i} + k_B \ln Z \cancelto{\sum_i P_i =1}{\sum_i \frac{e^{-\beta E_i}}{Z}} \\ S &= k_B \beta U + k_B \ln Z \\ U &= \frac{S}{k_B\beta} - \frac1{\beta} \ln Z \label{eq:U} \end{align}\]

    I'll note that I would normally not show so many steps above in my own work, but am being extra thorough in this solution.

    We are now at the point where we can start thinking thermo, and make use of the thermodynamic identity. To do that, let us "zap with d" to see what \(dU\) is in terms of \(dS\) and \(d\beta\), etc. \[\begin{align} dU &= TdS - p\cancelto{0}{dV } \\ &= \frac{dS}{k_B\beta} - \frac{S}{k_B\beta^2}d\beta + \frac{\ln Z}{\beta^2}d\beta -\frac1{\beta Z}dZ \end{align}\]

    So far, this may not look promising to you, but perseverence pays off!

    I threw out the \(dV\) because our statistical formalism only includes states with a given volume. Including the volume dependence is not complicated, but it requires us to take derivatives of \(E_i\) with respect to volume, which is a nuisance we can live without for now.
    Let's begin by lumping together the two \(d\beta\) terms. They look suspiciously similar to our previous expression for \(U\), which is unshocking, since Eq. \(\ref{eq:U}\) showed that \(U\) was inversely proportional to \(\beta\). \[\begin{align} dU &= \frac{dS}{k_B\beta} - \left(\frac{S}{k_B\beta} -\frac{\ln Z}{\beta}\right)\frac{d\beta}{\beta} -\frac1{\beta Z}dZ \\ &= \frac{dS}{k_B\beta} - \frac{U}{\beta}d\beta -\frac1{\beta Z}dZ \end{align}\] Let's start by unpacking this \(dZ\): \[\begin{align} dZ &= \sum_i e^{-\beta E_i} \left(-E_i d\beta \right) \\ &= -d\beta \sum_i e^{-\beta E_i} E_i \\ &= -Z d\beta \sum_i \frac{e^{-\beta E_i}}{Z} E_i \\ &= -Z U d\beta \end{align}\] Yay for recognizing something we have computed before! Now let's put this back in our \(dU\). \[\begin{align} dU &= \frac{dS}{k_B\beta} - \frac{U}{\beta}d\beta -\frac1{\beta Z} (-ZUd\beta) \\ &= \frac{dS}{k_B\beta} \end{align}\] Whew! Everything cancelled (as it had to, but one algebra error would mess this all up...), and we are left with a simple expression that does not have a \(d\beta\) in sight! This is good, because we argued before that with volume held fixed \[\begin{align} dU &= TdS = \frac{1}{k_B\beta}dS \\ T &= \frac{1}{k_B\beta} \\ \beta &= \frac1{k_BT} \end{align}\]

    So we have just proven what you all knew about the relationship between \(\beta\) and \(T\). This is valuable, because it establishes the connection between the theoretical Lagrange multiplier and the temperature defined in thermodynamics.

    The text uses a different (microcanonical) approach to establishing the connection between the statistical approach and the temperature that we define in thermodynamics.

  2. Gibbs entropy is extensive

    1. To begin, we remember the relationship between probabilities given in the problem: \[\begin{align} P_{ij}^{AB} &= P_i^AP_j^B \end{align}\] This means that the probability of finding system \(A\) in state \(i\) while system \(B\) is in state \(j\) is just the product of the separate probabilities. Now the entropy is \[\begin{align} S_{AB} &= \sum_\alpha^{\text{all states}} P_\alpha \ln P_\alpha \\ &= \sum_i^{\text{states of $A$}}\sum_j^{\text{states of $B$}} P_i^AP_j^B \ln\left(P_i^AP_j^B\right) \\ &= \sum_i^{\text{states of $A$}}\sum_j^{\text{states of $B$}} P_i^AP_j^B \left(\ln P_i^A + \ln P_j^B\right) \\ &= \sum_i\sum_j P_i^AP_j^B\ln P_i^A + \sum_i\sum_j P_i^AP_j^B \ln P_j^B \\ &= \sum_i P_i^A \ln P_i^A\sum_j P_j^B + \sum_i P_i^A\sum_j P_j^B \ln P_j^B \\ &= \cancelto{S_A}{\left(\sum_i P_i^A \ln P_i^A\right)}\cancelto{1}{\left(\sum_j P_j^B\right)} + \cancelto{1}{\left(\sum_i P_i^A\right)}\cancelto{S_B}{\left(\sum_j P_j^B \ln P_j^B\right)} \\ &= S_A+S_B \end{align}\]
    2. At this stage, we can basically use just words to solve this. We consider the system of \(N\) identical subsystems as a combination of a single one and \(N-1\) subsystems, thus \(S_N = S_1 + S_{N-1}\) from what we showed above. Then we repeat \(N\) times, to show that \(S_N=NS_1\). There are other ways to show this, e.g. by repeatedly dividing the system (\(S_N=S_{N/2}+S_{N/2}\)).
  3. Boltzmann probabilities

    1. At infinite temperature \(\beta=0\), which makes computing probabilities easy: they are all equal. Thus the probabilities are each \(1/3\).

    2. At very low temperatures \(\beta\epsilon\gg1\). Remember that the probabilities are given by \[\begin{align} P_i &= \frac{e^{-\beta E_i}}{Z} \\ Z &= e^{\beta\epsilon} + 1 + e^{-\beta\epsilon} \end{align}\] We can see that our "small quantity" for a power series should be \(e^{-\beta\epsilon}\), since that is the small thing in the partition function. We can start with the ground state, which we expect to be overwhelmingly occupied: \[\begin{align} P_{0} &= \frac{e^{\beta\epsilon}}{e^{\beta\epsilon} + 1 + e^{-\beta\epsilon}} \\ &= \frac{1}{1 + e^{-\beta\epsilon} + e^{-2\beta\epsilon}} \\ &\approx 1 - \left(e^{-\beta\epsilon} + \cancel{e^{-2\beta\epsilon}}\right) + \left(\cancel{e^{-\beta\epsilon}} + e^{-2\beta\epsilon}\right)^2 + \cdots \end{align}\] At the last step, we used a power series approximation for \(1/(1-z)\). We now need to gather terms so that we keep all terms to the same order. In this case the best option is to keep all terms up to \(e^{-2\beta\epsilon}\), since that way we will be able to account for the occupation of the highest energy state. Keeping these terms gives \[\begin{align} P_0 \approx 1 - e^{-\beta\epsilon} + \mathcal{O}\left(e^{-3\beta\epsilon}\right) \end{align}\] becaue the \(e^{-2\beta\epsilon}\) terms cancel each other out. Thus the ground state will be almost 100% occupied. When we look at the other two states we will get exponentially smaller probabilities: \[\begin{align} P_{1} &= \frac{1}{e^{\beta\epsilon} + 1 + e^{-\beta\epsilon}} \\ &= e^{-\beta\epsilon}\frac{1}{1 + e^{-\beta\epsilon} + e^{-2\beta\epsilon}} \\ &= e^{-\beta\epsilon} P_0 \\ &\approx e^{-\beta\epsilon}\left(1 - e^{-\beta\epsilon}\right) \\ &= e^{-\beta\epsilon} - e^{-2\beta\epsilon} \end{align}\] The middle state with zero energy is less occupied by precisely a factor of \(e^{-\beta\epsilon}\). We could have predicted this from the Boltzmann ratio. \[\begin{align} P_{2} &= \frac{e^{-\beta\epsilon}}{e^{\beta\epsilon} + 1 + e^{-\beta\epsilon}} \\ &= e^{-\beta\epsilon}P_1 \\ &\approx e^{-2\beta\epsilon} \end{align}\]

      And the high energy state is hardly occupied at all, the same factor smaller than the previous state.

      This solution kept all terms that were at least order \(e^{-2\beta\epsilon}\) for each probability, which resulted in a set of probabilities that add up to one. It would also have been reasonable to answer that \(P_0\approx 1\) and \(P_1\approx e^{-\beta\epsilon}\), and then discuss that actually the probability of being in the ground state is not precisely 1.

      I could understand saying that \(P_2\approx 0\), but ideally you should give a nonzero answer for each probability when asked about very low temperatures, because none of them are exactly zero. If you have an experimant that measures \(P_2\) (perhaps state 2 has a distinctive property you can observe), then you will not find it to be zero at any temperature (unless you have poor resolution), and it is best to show how it scales.

    3. If we allow the temperature to be negative, then higher energy states will be more probable than lower energy states. If the energy is small and negative (which was not specified in the question), then the system will almost always be in the \(+\epsilon\) energy state.

    Another behavior with negative temperatures for this system is that \(U>0\). For positive temperatures, the internal energy only approaches zero as the temperature gets very high. If the temperature becomes negative, the energy can exceed zero. For other systems, of course, this will not be the case, but this will be true for any system in which the energy states are symmetrically arranged around zero.

Solution for week 2

PDF version of solutions

  1. Entropy and Temperature (K&K 2.1)

    1. We begin by finding the entropy given the provided multiplicity. \[\begin{align} S(U,N) &= k_B\log g(U,N) \\ &= k_B \log \left(CU^{3N/2}\right) \\ &= k_B\left( \log C + \log \left(U^{3N/2}\right) \right) \\ &= k_B\log C + \frac32 Nk_B \log U \\ \end{align}\] In the last two steps there, we made use of properties of \(\log\). If these are not obvious to you, you absolutely must take the time to review the properties of logarithms. They are absolutely critical to this course! \[\begin{align} \frac1{T} &= \left(\frac{\partial S}{\partial U}\right)_{V,N} \\ &= \frac32Nk_B\frac1{U} \\ U &= \frac32Nk_BT \end{align}\] Yay.
    2. We just need to take one more derivative, since we already found \(\left(\frac{\partial S}{\partial U}\right)_{V,N}\) in part (a). \[\begin{align} \left(\frac{\partial^2S}{\partial U^2}\right)_{V,N} &= -\frac32Nk_B\frac1{U^2} \\ &< 0, \end{align}\]

      where in the last step we only needed to assume that \(N>0\) (natural for the number of particles) and that the energy \(U\) is real (which it always must be). Thus ends the solution.

      Because the second derivative of the entropy is always negative, the first derivative is monotonic, which means that the temperature (which is positive) will always increase if you increase the energy of the system and vice versa.

  2. Paramagnetism (K&K 2.2)

    Okay, here we can understand the fractional magnetization if we think \(Nm\) as being the maximum possible magnetization (all spins are pointing the same way). The quantity \(s\) is defined as the total value of the spin. Because each spin has a value of \(\pm\frac12\), twice \(s\) per particle also tells us the fractional magnetization.

    To convert from \(S(s)\) to \(S(U)\) we need to relate the energy to the excess spin \(s\). This relies on the energy expression \[\begin{equation} U = -B2sm \end{equation}\] which uses the equations given. At this point, it is a simple substitution of \(s=-\frac{U}{2mB}\): \[\begin{align} S(U) &= S_0 - k_B\frac{2\left(-\frac{U}{2mB}\right)^2}{N} \\ &= S_0 - k_B\frac{U^2}{2m^2B^2N} \end{align}\] To determine \(1/kT\), we just need to take a derivative: \[\begin{align} \frac1{T} &= \left(\frac{\partial S}{\partial U}\right)_V \\ &= -k_B\frac{U}{m^2B^2N} \\ \frac1{k_BT} &= -\frac{U}{m^2B^2N} \end{align}\]

    At this point we have finished the problem, with just a bit of algebra. It is helpful at this stage to do a bit of checking. The left hand side here is an inverse energy. On the right hand side, \(mB\) is an energy, so we have an energy over an energy squared, so all is good. \(N\), of course, is dimensionless. However, it is also extensive, as is \(U\). This is good, because the left hand side is intensive, so the right hand side should be also.

    Of interest

    This relationship is very different than the one we saw in the previous problem! Previously, we saw the temperature being proportional to the internal energy, and here we see it as inversely proportional, meaning that as the energy approaches zero the temperature becomes infinite.

    We also previously had an energy that was positive. Here we have a negative sign, which suggests that the energy should be negative in order to maintain a positive temperature. This relates to the energy of a single spin always being either positive or negative, with equal and opposite options.

    This problem illustrates a weird phenomenon: if the energy is positive, then we must conclude that the temperature is negative. Furthermore, the temperature discontinuously passes from \(\infty\) to \(-\infty\) as the energy passes through zero. There are different interpretations of these "negative temperature" states. You cannot reach them by heating a system (adding energy via heating), and they cannot be exist in equilibrium if the system has contact with any quantity of material that can have kinetic energy. So I consider these to be unphysical (or non-equilibrium) states. Since temperature is an equilibrium proprty, I would not say that a negative temperature is physically meaningful. That said, there is an analogy that can be made to population inversion in a laser, which is a highly non-equilibrium system that is pretty interesting.

  3. Quantum harmonic oscillator (K&K 2.3)
    1. Given the multiplicity, we just need to take a logarithm, and simplify. \[\begin{align} S(N,n) &= k\log g(N,n) \\ &= k \log\left(\frac{(N+n-1)!}{n!(N-1)!}\right) \\ &= k\left(\log(N+n+1)! - \log n! - \log(N-1)!\right) \\ &\approx k\left((N+n+1)\log(N+n+1) - n\log n - (N-1)\log(N-1)\right) \\ &\approx k\left((N+n)\log(N+n) - n\log n - N\log N\right) \\ &= k\left(N\log\frac{N+n}{N} + n\log\frac{N+n}{n}\right) \\ &= k\left(N\log\left(1+n/N\right) + n\log\left(1+N/n\right)\right) \end{align}\] You need not simplify your answer this far, but it is good to get practice simplifying answers, particularly involving logarithms. In particular, it is usually helpful at this point in a computation to verify that the entropy is indeed extensive. Both \(N\) (the number of oscillators) and \(n\) (the sum of all the quantum numbers of all the oscillators) are extensive quantities. Thus \(n/N\) and \(N/n\) are intensive, which is good because otherwise we could not add them to \(1\). Each term is now clearly extensive, and the entropy behaves as we expect.
    2. Now we want to find \(U(T)\), which will require us to find \(S(U)\) (via simple substitution of \(n=U/\hbar\omega\)) and \(T\) from a derivative of that. \[\begin{align} S(U) &= Nk\log\left(1+\frac{U}{N\hbar\omega}\right) + Nk\frac{U}{N\hbar\omega}\log\left(1+\frac{N\hbar\omega}{U}\right) \end{align}\]

      Now we just have a derivative to take, and then a mess of algebra to simplify.

      \[\begin{align} \frac1{T} &= \left(\frac{\partial S}{\partial U}\right)_{N,V} \\ &= \frac{Nk}{1+\frac{U}{N\hbar\omega}}\frac{1}{N\hbar\omega} \\&\quad + Nk\frac{1}{N\hbar\omega}\log\left(1+\frac{N\hbar\omega}{U}\right) \\&\quad - Nk\frac{U}{N\hbar\omega}\frac{1}{1+\frac{N\hbar\omega}{U}}\frac{N\hbar\omega}{U^2} \end{align}\] And now to simplify... \[\begin{align} \frac{\hbar\omega}{kT} &= \frac{1}{1+\frac{U}{N\hbar\omega}} + \log\left(1+\frac{N\hbar\omega}{U}\right) - \frac{\frac{N\hbar\omega}{U}}{1+\frac{N\hbar\omega}{U}} \\ &= \frac{\frac{N\hbar\omega}{U}}{1+\frac{N\hbar\omega}{U}} + \log\left(1+\frac{N\hbar\omega}{U}\right) - \frac{\frac{N\hbar\omega}{U}}{1+\frac{N\hbar\omega}{U}} \\ &= \log\left(1+\frac{N\hbar\omega}{U}\right) \end{align}\]

      Well, didn't that simplify down nicely? The key was to multiply the first term by \(N\hbar\omega/U\) so that it shared a denominator with the last term (and ended up being equal and opposite).

      Solving for \(U\) is not bad at all, now, we'll just take an exponential of both sides: \[\begin{align} e^{\frac{\hbar\omega}{kT}} &= 1 + \frac{N\hbar\omega}{U} \\ \frac{N\hbar\omega}{U} &= e^{\frac{\hbar\omega}{kT}} - 1 \\ U &= \frac{N\hbar\omega}{e^{\frac{\hbar\omega}{kT}} - 1} \end{align}\]
      As I mentioned in the homework, this is the hard way to solve this problem. That said, it wasn't actually particularly hard, you just need to be comfortable doing algebra with logarithms, and simplifying annoying ratios.

Solution for week 3

PDF version of solutions

  1. Free energy of a two state system (K&K 3.1, modified)
    1. The partition function of this two-state system is very simple: \[\begin{align} Z &= \sum_{s}^{\text{all states}} e^{-\beta E_s} \\ &= e^{0} + e^{-\beta\varepsilon} \\ &= 1 + e^{-\beta\varepsilon} \end{align}\] Now the free energy is just a log of this: \[\begin{align} F &= -kT\log Z \\ &= -kT\log\left(1 + e^{-\beta\varepsilon}\right) \\ &= -kT\log\left(1 + e^{-\frac{\varepsilon}{kT}}\right) \end{align}\] We can ask ourselves if this simplifies in any limits, and the easiest one is the low-temperature limit where \(e^{-\frac{\varepsilon}{kT}}\ll 1\). In this limit, the free energy is given by \[\begin{align} F &\approx -kT e^{-\frac{\varepsilon}{kT}} \end{align}\]
    2. From the free energy, find expressions for the internal energy \(U\) and entropy \(S\) of the system.
    3. Plot the entropy versus \(T\). Explain its asymptotic behavior as the temperature becomes high.
    4. Plot the \(S(T)\) versus \(U(T)\). Explain the maximum value of the energy \(U\).
  2. Magnetic susceptibility (K&K 3.2)
    1. Use the partition function to find an exact expression for the magentization \(M\) and the susceptibility \(\chi\equiv\left(\frac{\partial M}{\partial B}\right)_T\) as a function of temperature and magnetic field for the model system of magnetic moments in a magnetic field. The result for the magnetization is \(M=nm\tanh\left(\frac{mB}{kT}\right)\), as derived in (46) by another method. Here \(n\) is the particle concentration. The result is plotted in the book.
    2. Find the free energy and epress the result as a function only of \(T\) and the parameter \(x\equiv \frac{M}{nm}\).
    3. Show that the susceptibility is \(\chi=\frac{nm^2}{kT}\) in the limit \(mB\ll kT\).
  3. Free energy of a harmonic oscillator (K&K 3.3) A one-dimensional harmonic oscillator has an infinite series of equally spaced energy states, with \(\varepsilon_s = s\hbar\omega\), where \(s\) is a positive integer or zero, and \(\omega\) is the classical frequency of the oscillator. We have chosen the zero of energy at the state \(s=0\) [which Kittel can get away with here, but is not actually the zero of energy!].
    1. Show that for a harmonic oscillator the free energy is \[\begin{equation} F = k_BT\log\left(1 - e^{-\frac{\hbar\omega}{k_BT}}\right) \end{equation}\] Note that at high temperatures such that \(k_BT\gg\hbar\omega\) we may expand the argument of the logarithm to obtain \(F\approx k_BT\log\left(\frac{\hbar\omega}{kT}\right)\).
    2. From the free energy above, show that the entropy is \[\begin{equation} S = \frac{\frac{\hbar\omega}{kT}}{e^{\frac{\hbar\omega}{kT}}-1} - \log\left(1-e^{-\frac{\hbar\omega}{kT}}\right) \end{equation}\] This entropy is shown in the book, along with the heat capacity. FIXME I could put those plots here, along with their captions...
  4. Energy fluctuations (K&K 3.4) Consider a system of fixed volume in thermal contact with a resevoir. Show that the mean square fluctuations in the energy of the system is \[\begin{equation} \left<(\varepsilon-\langle\varepsilon^2\rangle\right> = k_BT^2\left(\frac{\partial U}{\partial T}\right)_{V} \end{equation}\] Here \(U\) is the conventional symbol for \(\langle\varepsilon\rangle\). Hint: Use the partition function \(Z\) to relate \(\left(\frac{\partial U}{\partial T}\right)_V\) to the mean square fluctuation. Also, multiply out the term \((\cdots)^2\). Note: The temperature \(T\) of a system is a quantity that by definition does not fluctuate in value when the system is in thermal contact with a resevoir. Any other attitude would be inconsistent with our definition of the temperature of a system. The energy of such a system may fluctuate, but the temperature does not. Some workers do not adhere to a rigorous defintion of temperature. Thus Landau and Lifshitz give the result \[\begin{align} \left<(\Delta T)^2\right> = \frac{k_BT^2}{C_V}, \end{align}\]

    but this should be viewed as just another form of what you have proven in this problem, with \(\Delta T\) set equal to \(\Delta U/C_V\). We know that \(\Delta U = C_V\Delta T\) (for small values of \(\Delta T\)), whence these two equations become equal.

  5. Quantum concentration (K&K 3.8) We need to start by finding the formula for the ground state energy of a particle in a box. It's all right if you just remember this, but it also isn't hard to figure out without doing any complicated math or boundary conditions. The particle needs to have a half-wavelength of \(L\) in each direction in order to fit in the box, thus \(\lambda = 2L\) for each direction. This means that that the wave vector is given by \(k_x = k_y = k_z = \pm\frac{2\pi}{2L}\). Of course, the particle isn't in a traveling wave, but rather in a superposition of the \(\pm\) versions of these traveling waves (i.e. a standing wave). The kinetic energy is given by \[\begin{align} KE &= \frac{p^2}{2M} \\ &= \frac{\hbar^2k^2}{2M} \\ &= \frac{\hbar^2\pi^2}{2ML^2} \end{align}\] Now as instructed we set the kinetic energy to \(kT\) and solve for the density, given by \(n=\frac1{L^3}\). \[\begin{align} kT &= \frac{\hbar^2\pi^2}{2ML^2} \\ &= \frac{\hbar^2\pi^2}{2Mn^{-2/3}} \\ n^{-\frac23} &= \frac{\hbar^2\pi^2}{2MkT} \\ n &= \left(\frac{2MkT}{\hbar^2\pi^2}\right)^\frac32 \end{align}\]

    As predicted, this difers from \(n_Q\) by a factor of \(\left(\frac{4}{\pi}\right)^\frac32\), which is of order unity. Its value is around \(1.4\).

    Let's remind ourselves: this is the density at which quantum stuff becomes really important at a given temperature. In this problem, we showed that the density \(n_Q\) is basically the same as the density of a single particle that is confined enough that its kinetic energy is the same as the temperature.

  6. One-dimensional gas (K&K 3.11) Consider an ideal gas of \(N\) particles, each of mass \(M\), confined to a one-dimensional line of length \(L\). Find the entropy at temperature \(T\). The particles have spin zero.

    To find the entropy at temperature \(T\) we need first to consider the energy eigenstates of this system. We could use either periodic boundary conditions or a particle-in-a-box potential to confine the particles. The kinetic energy of a plane wave is given by \[\begin{align} E_k &= \frac{\hbar^2k^2}{2m} \end{align}\] For a particle in a box, a half-integer number of wavelengths must fit in the box: \[\begin{align} L &= 2n\lambda \\ &= 2n\frac{2\pi}{k} \\ k_n &= \frac{4\pi n}{L}. \end{align}\] Thus, the energy is given by \[\begin{align} E_n &= \frac{\hbar^2}{2m}\frac{16\pi^2 n^2}{L^2} \\ &= \frac{8\pi^2\hbar^2 n^2}{mL^2} \end{align}\]

    No, you don't need to derive this, but in my opinion it is easier to derive than to remember. If I were not writing up solutions, I would have done several of the steps above in my head.

    Now that we have our energy, we can start thinking about how to find the partition function for a single particle. We can get away with this because the particles are non-interacting, so the total energy is just a sum of the energies of each individual particle. \[\begin{align} Z_1 &= \sum_{n=1}^{\infty} e^{-\beta E_n} \\ &= \sum_{n=1}^{\infty} e^{-\beta \frac{8\pi^2\hbar^2}{mL^2} n^2} \end{align}\] This is the point where we typically step back and tell ourselves that \(k_BT \gg \frac{8\pi^2\hbar^2}{mL^2}\) (because the distance \(L\) is macroscopic, and we aren't looking at crazy-low temperatures), which means that we can turn the sum into an integral: \[\begin{align} Z_1 &\approx \int_{0}^{\infty} e^{-\beta \frac{8\pi^2\hbar^2}{mL^2} n^2} dn \end{align}\] The smart move here it to do a \(u\) substitution, to get all the ugly stuff out of our integral. \[\begin{align} u &= \sqrt{\frac{2\beta}{m}}\frac{2\pi\hbar}{L} n & du &= \sqrt{\frac{2\beta}{m}}\frac{2\pi\hbar}{L} dn \end{align}\] This gives us a simple gaussian integral: \[\begin{align} Z_1 &\approx \sqrt{\frac{mkT}{2}}\frac{L}{2\pi\hbar} \int_{0}^{\infty} e^{-u^2} du. \end{align}\]

    The value of the gaussian integral here doesn't have any particular physical impact, since it is just a dimensionless numerical constant. It does, of course, impact the numerical value.

    Gaussian integrals
    You are welcome to look up the value of integrals like this, or memorize the answer (I always just remember that it's got a \(\sqrt{\pi}\) in it, which doesn't help much). I'll show you here how to find the value of a gaussian integral, which is a nice trick to be aware of. \[\begin{align} I_G &\equiv \int_{0}^{\infty} e^{-u^2} du \\ &= \frac12 \int_{-\infty}^{\infty} e^{-u^2} du \\ (2I_G)^2 &= \left(\int_{-\infty}^{\infty} e^{-u^2} du\right)^2 \\ &= \left(\int_{-\infty}^{\infty} e^{-x^2} dx\right)\left(\int_{-\infty}^{\infty} e^{-y^2} dy\right) \\ &= \int_{-\infty}^{\infty}\int_{-\infty}^{\infty} e^{-\left(x^2+y^2\right)} dxdy \\ &= \int_{0}^{\infty}\int_{0}^{2\pi} e^{-r^2} r d\phi dr \\ &= 2\pi \int_{0}^{\infty} e^{-r^2} r dr \\ \xi &= r^2\quad\quad d\xi = 2rdr \\ (2I_G)^2 &= 2\pi \int_0^\infty e^{-\xi}\frac{d\xi}{2} \\ &= \pi \\ I_G &= \frac{\sqrt{\pi}}2 \end{align}\] The trick was simply to square the integral, and then move from Cartesian to polar coordinates. This only works because we are integrating to \(\infty\).
    Back to our original task, we have now found that \[\begin{align} Z_1 &\approx \sqrt{\frac{\pi mkT}{2}}\frac{L}{4\pi\hbar} \end{align}\] To find the entropy, we will want to construct the Helmholtz free energy. We will need the entire partition function, which will have the \(N!\) in it to avoid double-counting states, since these are indistinguishable particles. \[\begin{align} Z &= \frac{Z_1^N}{N!} \\ F &= -k_BT \log Z \\ &= -k_BT \log\left(\frac{Z_1^N}{N!}\right) \\ &= -k_BT\left(N\log Z_1 - \log N!\right) \\ &\approx -k_BT\left(N\log Z_1 - N\log N + N\right) \\ &= -Nk_BT\left(\log Z_1 - \log N + 1\right) \\ &= -Nk_BT\left(\log \left(\sqrt{\frac{\pi mkT}{2}}\frac{L}{4\pi\hbar}\right) - \log N + 1\right) \\ &= -Nk_BT\left(\log \left(\sqrt{\frac{m kT}{32\pi \hbar^2}}\frac{L}{N}\right) + 1\right) \\ \end{align}\]

    Here we are at a good point to check whether our answer makes sense. Firstly, we can check that \(F\) is indeed extensive. It is, since it is proportional to \(N\), and the only other extensive quantities in it are the \(L\) and \(N\) in the logarithm, and they form a ratio, which is therefore intensive. We can also check dimensions.

    We know that \(\frac{\hbar^2k^2}{2m}\) is an energy, which means that \(\frac{\hbar^2}{m}\) has dimensions of energy times distance squared. The \(kT\) cancels the energy, and the square root turns the resulting one over distance squared into an inverse distance, which happily cancels with the \(L\), so the argument of our logarithm is dimensionless.

    Now we will get to the answer pretty soon! Recall that: \[\begin{align} dF &= -SdT -pdV \end{align}\] (although the second term should have a \(dL\) for a 1D system) which means that \[\begin{align} -S &= \left(\frac{\partial F}{\partial T}\right)_{L,V,N} \\ S &= -\left(\frac{\partial F}{\partial T}\right)_{L,V,N} \\ &= Nk_B\left(\log \left(\sqrt{\frac{m kT}{32\pi \hbar^2}}\frac{L}{N}\right) + 1\right) + Nk_BT\frac{1}{2T} \\ &= Nk_B\left(\log \left(\sqrt{\frac{m kT}{32\pi \hbar^2}}\frac{L}{N}\right) + \frac32\right) \end{align}\]

    This is our final answer. It looks shockingly like the entropy of a 3D ideal gas, right down to the quantum length scale (which is no longer a quantum density), commonly called the "thermal de Broglie wavelength."

    Nasty logs
    In computing the derivative of a nasty logarithm (which I did in my head above), you can use the following shortcut, provided you remember the properties of logarithms: \[\begin{align} \log\left(\sqrt{\frac{m kT}{32\pi \hbar^2}}\frac{L}{N}\right) &= \log\left(\sqrt{\frac{m kT}{32\pi \hbar^2}}\right) + \log\left(\frac{L}{N}\right) \\ &= \frac12\log\left(\frac{m kT}{32\pi \hbar^2}\right) + \log\left(\frac{L}{N}\right) \\ &= \frac12\log\left(T\right) + \frac12\log\left(\frac{m k}{32\pi\hbar^2}\right) + \log\left(\frac{L}{N}\right) \end{align}\]

    Now you can take a derivative of this, which is way easier than a derivative of the whole mess, and clearly must give you the same answer. There are other ways to do it, but I find this particularly simple, and it has the advantage that it's the same kind of manipulation you're doing all the time anyhow, just to simplify your results.

    If you do this in your head (or on scratch paper), you can immediately discard any of the terms that will vanish when you take a derivative of them, which makes it much simpler than what I wrote down.

Solution for week 4

PDF version of solutions

  1. Number of thermal photons We are looking to find the number of thermal photons in a cavity with volume \(V\). The key is that we already know how many photons are in each mode, we just have to add up the photons from every mode. It may sound tedious, but that is only because it is.

    For a simple harmonic oscillator, the number of quanta is given by \[\begin{align} \langle n\rangle &= \frac{e^{-\beta\hbar\omega}}{1 - e^{-\beta\hbar\omega}} \\ &= \frac{1}{e^{\beta\hbar\omega}-1} \end{align}\] And this is what we need to add up for all the modes. \[\begin{align} N_{ph} &= \sum_{n_x=-\infty}^\infty\sum_{n_y}\sum_{n_z} \frac{1}{e^{\beta\hbar\omega_{n_xn_yn_z}}-1} \\ &= \sum_{n_x=-\infty}^\infty\sum_{n_y}\sum_{n_z} \frac{1}{e^{\beta \frac{hc}{L}\sqrt{n_x^2+n_y^2+n_z^2}}-1} \end{align}\] As in class, we will turn our summation into an integral, assuming our volume is reasonably large (and our temperature is reasonably high) so \(\frac{hc}{kTL} \ll 1\). \[\begin{align} N_{ph} &\approx \iiint \frac{1}{e^{\beta \frac{hc}{L}\sqrt{n_x^2+n_y^2+n_z^2}}-1} dn_xdn_ydn_z \\ &= \int_0^\infty \frac{1}{e^{\beta \frac{hc}{L}n}-1}4\pi n^2 dn \end{align}\] We will do a \(\xi\) substitution: \(\xi = \frac{\beta hc}{L}n\) \[\begin{align} N_{ph} &= \left(\frac{kTL}{hc}\right)^3 \int_0^\infty \frac1{e^\xi-1}\xi^2d\xi \\ \frac{N_{ph}}{V} &= \left(\frac{kT}{hc}\right)^3 \int_0^\infty \frac1{e^\xi-1}\xi^2d\xi \end{align}\]

    Once again, we have a dimensionless integral that we can look up the value of, but which only scales our answer. Wolfram alpha says the value of this integral is about 2.4. I don't think this quite agrees with Kittel's answer, but am going to move on to the next solution for now.

  2. Surface temperature of the sun

    1. Power of the sun Given the flux of power hitting the earth, we just need to find the fraction of the sun's power that hits a square centimeter on the earth. That is just the inverse of the area of a sphere with radius equal to the earth-sun distance. \[\begin{align} \text{power} &= \frac{4\pi R^2}{1 \text{cm}^2} 0.136 \text{J}\text{s}^{-1}\text{cm}^{-2} \\ &= \frac{4\pi \left(1.5\times10^{13}\text{cm}\right)^2}{1\text{cm}^2} 0.136 \text{J}\text{s}^{-1}\text{cm}^{-2} \\ &\approx 3.85\times 10^{26} \text{J}/\text{s} \end{align}\]
    2. Temperature The power radiated by the sun is given by the Stefan-Boltzmann constant times the area of the sun times \(T^4\). Thus \[\begin{align} T &= \frac{\text{power}}{4\pi R^2 \sigma_{SB}} \end{align}\] FIXME look up the constant value and do math!

    I'll note that this is a weird way to find the temperature of the sun. The easier approach is to use the frequency with maximum intensity, but of course that requires different information.

  3. Age of the sun

    1. The total energy available in the sun in this approximation is \[\begin{align} E &\approx 0.1(M_\text{H} - M_\text{He})c^2 \end{align}\] Where by \(M_\text{H}\) I mean the mass of the sun when it is all hydrogen, and by \(M_\text{He}\) I mean the mass of the sun when it is all helium. We could compute the mass of the sun using Newton's law of gravity, the distance of the earth from the sun, and the length of a year. Or we could look it up. Either way it comes out to \(\sim 2\times 10^{30}~\text{kg}\). Thus our answer is \[\begin{align} E &\approx 0.1\frac{4\times 1.0078 - 4.0026}{4\times 1.0078}\times 10^{30}~\text{kg} c^2 \\ &\approx 0.1\frac{4.0312 - 4.0026}{4.0312}\times 10^{30}~\text{kg} c^2 \\ &\approx 0.00071\times 2\times 10^{30}~\text{kg} c^2 \\ &\approx 1.4\times 10^{27}~\text{kg} c^2 \\ &\approx 1.3\times 10^{44}~\text{J} \end{align}\]
    2. To find the lifetime, we just divide the energy by the power, assuming it keeps radiating at the same rate. \[\begin{align} t_{\max} &\approx \frac{1.3\times 10^{44}\text{J}}{4\times 10^{26}\text{Js}^{-1}} \\ &\approx 3\times 10^{17}\text{s} \\ &\approx 5\times 10^{15}~\text{min} \\ &\approx 10^{10}~\text{years} \end{align}\] So according to this estimate, our sun is basically done. Keep in mind the 10% number used was unjustified. A quick web search suggests that people think the sun can continue "as is" for about another 5 billion years, so there is no need to cash out on your Earth real estate just yet.
  4. Surface temperature of the earth

    This problem comes down to balancing the radiation absorbed by the earth with the radiation emitted by the earth. Interestingly, the answer won't change if we drop the assumption that the earth is a perfect black body, so long as its absorption is indepednent of frequency (which isn't true). The assumption that it remains constant temperature over day and night is also a bit weak, but fractionally the variation of temperature is actually relatively small.

    The total energy radiated by the sun is \[\begin{align} P_{\odot} &= \sigma_{SB} T_{\odot}^4 4\pi R_{\odot}^2 \end{align}\] Now, the fraction \(f\) of that power that is absorbed by the Earth is given by \[\begin{align} f &= \frac{\text{cross-sectional area of earth}}{\text{area of sphere with Earth's orbit}} \\ &= \frac{\pi R_E^2}{4\pi AU^2} \end{align}\] Okay, now we just need the energy radiated by the earth: \[\begin{align} P_E &= \sigma_{SB} T_{E}^4 4\pi R_{E}^2 \end{align}\] Setting the energy absorbed by the earth to the energy radiated by the earth gives \[\begin{align} \frac{\pi R_E^2}{4\pi AU^2}\cancel{\sigma_{SB}} T_{\odot}^4 \bcancel{4\pi} R_{\odot}^2 &= \cancel{\sigma_{SB}} T_{E}^4 \bcancel{4\pi} R_{E}^2 \\ T_E^4 &= \frac{R_E^2}{4 AU^2} T_{\odot}^4\frac{R_{\odot}^2}{R_E^2} \\ &= \frac14 T_{\odot}^4\frac{R_{\odot}^2}{AU^2} \end{align}\] \[\begin{align} T_E &= \sqrt{\frac{R_{\odot}}{2 AU}}T_{\odot} \\ &= \sqrt{\frac{7\times 10^{10}\text{cm}}{3\times 10^{13}\text{cm}}}5800\text{K} \\ &\approx 280\text{K} \end{align}\]

    This is a bit cold, but when you consider the approxmations isn't a bad estimate of the Earth's temperature. This neglects the power from radioactivity, and also the greenhouse effect (which is a violation of the assumption that the absorption and emmission have the same proportion at all wavelengths).

  5. Pressure of thermal radiation

  6. Heat shields

Solution for week 5

PDF version of solutions

  1. First question.

  2. Second question.

  3. Third question.

Solution for week 6

PDF version of solutions

  1. First question.

  2. Second question.

  3. Third question.

Solution for week 7

PDF version of solutions

  1. First question.

  2. Second question.

  3. Third question.

Solution for week 8

PDF version of solutions

  1. First question.

  2. Second question.

  3. Third question.

Solution for week 9

PDF version of solutions

  1. First question.

  2. Second question.

  3. Third question.

  1. Of course, we already talked last week about \(F=-kT\ln Z\), but that was done using the Gibbs entopy, which we're pretending we don't yet know...