LECTURE 9

Statistical Mechanics

Basic Methods

We have talked about ensembles being large collections of copies or clones of a system with some features being identical among all the copies. There are three different types of ensembles in statistical mechanics.

1. If the system under consideration is isolated, i.e., not interacting with any other system, then the ensemble is called the microcanonical ensemble. In this case the energy of the system is a constant.
2. If the system under consideration is in thermal equilibrium with a heat reservoir at temperature $T$, then the ensemble is called a canonical ensemble. In this case the energy of the system is not a constant; the temperature is constant.
3. If the system under consideration is in contact with both a heat reservoir and a particle reservoir, then the ensemble is called a grand canonical ensemble. In this case the energy and particle number of the system are not constant; the temperature and the chemical potential are constant. The chemical potential is the energy required to add a particle to the system.

The most common ensemble encountered in doing statistical mechanics is the canonical ensemble. We will explore many examples of the canonical ensemble. The grand canonical ensemble is used in dealing with quantum systems. The microcanonical ensemble is not used much because of the difficulty in identifying and evaluating the accessible microstates, but we will explore one simple system (the ideal gas) as an example of the microcanonical ensemble.

Microcanonical Ensemble

Consider an isolated system described by an energy in the range between $E$ and $E+\delta E$, and similar appropriate ranges for external parameters $x_{\alpha}$. To illustrate a microcanonical ensemble, consider only the energy parameter. Let $E_r$ be the total energy of the $r$th microstate. Also let $P_r$ be the probability of the system being in the $r$th microstate. The average energy is

$$\overline{E}=\sum_{r}P_r\;E_r$$ (1)

where the sum is over the accessible microstates. From the postulates of statistical mechanics that all microstates are equally probable, the probability of the system being in any microstate is a constant as long as the total energy is in the range $E$ to $E+\delta E$. Assume there are $\nu$ such states, then

$$P_r=\frac{1}{\nu}$$ (2)

The average value of any property of the system is

$$\overline{y}=\sum_{r}P_r\; y_r=\sum_{r}\frac{y_r}{\nu}$$ (3)

where $y_r$ is the value of the property $y$ when the system is in the $r$th microstate. The difficulty is that identifying the correct set of microstates is exceedingly difficult. If we think of phase space as consisting of all possible microstates of the system with all possible energies, then the microcanonical ensemble consists of the subset of phase space with microstates that have energy between $E$ and $E+\delta E$. Picking out these states is difficult. For example consider an ideal gas. Let each gas particle be a ``system''. Each system or particle is isolated and doesn't interact with anything. The microcanonical ensemble would consist of those particles with kinetic energy between $E$ and $E+\delta E$, i.e., it would consist of only those particles with a certain velocity. We could only sum over those particles, not all the particles. Picking out these particles is a pain.

Canonical Ensemble

The most common situation encountered in statistical mechanics is that of a system in thermal contact with a heat reservoir at constant temperature $T$. In equilibrium the system is also at temperature $T$. The system under consideration may be a small part of a larger system, for example, a 1 gram block of copper immersed in a container of liquid helium at 4.2 K.

Assume that system A is in thermal contact with a heat reservoir A$^{\prime}$. Thermal contact means that only heat can be exchanged between A and A$^{\prime}$. The energy of system A cannot be specified since it will fluctuate as heat is exchanged randomly between A and A$^{\prime}$ (but $\overline{E}$ will be well defined). Let $E_r$ be the energy of a microstate of A. Then

$$E_r+E^{\prime}=E^o$$ (4)

where $E^{\prime}$ is the energy of the heat reservoir A$^{\prime}$ and $E^o$ is the total energy of the combined system A and A$^{\prime}$. The probability $P_r$ of A being in microstate $r$ is proportional to the number $\Omega^{\prime}(E^o-E_r)$ of microstates of the reservoir:

$$P_r=C^{\prime}\Omega^{\prime}(E^o-E_r)$$ (5)

where $C^{\prime}$ is a constant determined by the normalization condition:

$$\sum_r P_r=1$$ (6)

Now assume $E_r\ll E^o$ (i.e., assume that A$^{\prime}$ is a heat reservoir) and expand about $E^{\prime}=E_o$:

$$\ln\Omega^{\prime}(E^o-E_r)=\ln\Omega^{\prime}(E^o)- \left.\... ...}(E^{\prime})}{\partial E^{\prime}} \right\vert _{E^o}E_r+...$$ (7)

But

$$\beta=\left.\frac{\partial\ln\Omega^{\prime}(E^{\prime})}{\partial E^{\prime}} \right\vert _{E^o}=\frac{1}{k_BT}$$ (8)

where $T$ is the temperature of the reservoir. Thus

$$\ln\Omega^{\prime}(E^o-E_r)=\ln\Omega^{\prime}(E^o)-\beta E_r$$ (9)

or

$$\Omega^{\prime}(E^o-E_r)=\Omega^{\prime}(E^o)e^{-\beta E_r}$$ (10)

Thus

$$P_r=C^{\prime}\Omega^{\prime}(E^o-E_r)=C^{\prime}\Omega^{\prime}(E^o) e^{-\beta E_r}=Ce^{-\beta E_r}$$ (11)

where

$$C=\frac{1}{\sum_r e^{-\beta E_r}}$$ (12)

Finally

$$P_r=\frac{e^{-\beta E_r}}{\sum_r e^{-\beta E_r}}$$ (13)

This probability distribution is sometimes called the Boltzmann distribution. It tells us the probability that a microstate with energy $E_r$ will be occupied. Notice that if $E_r<k_BT$, then there is a good chance that the state will be occupied. But if $E_r$ is large compared to the temperature, then the chance that the $r$th state is occupied is exponentially small.

The average value of any parameter $y$ is given by

$$\overline{y}=\sum_r P_r y_r=\frac{\sum_r y_re^{-\beta E_r}} {\sum_r e^{-\beta E_r}}$$ (14)

where $y_r$ is the value of the parameter $y$ in the $r$th state. For example, the mean energy is

$$\overline{E}=\frac{\sum_r E_re^{-\beta E_r}} {\sum_r e^{-\beta E_r}}$$ (15)

The denominator arises quite frequently. So let

$$Z\equiv\sum_r e^{-\beta E_r}$$ (16)

$Z$ is called the partition function. It acts like a generating function. For example,

$$\sum_r E_re^{-\beta E_r}=-\sum_r\frac{\partial}{\partial \be... ...beta}\sum_r e^{-\beta E_r}= -\frac{\partial Z}{\partial \beta}$$ (17)

or

$$\overline{E}=-\frac{1}{Z}\frac{\partial Z}{\partial \beta}= -\frac{\partial \ln Z}{\partial \beta}$$ (18)

The partition function $Z$ is quite useful and we can use it to generate all sorts of information about the statistical mechanics of the system.

The advantage of the canonical ensemble should now be apparent. The sum is over all the microstates of the system. We don't have the difficulty of finding only those microstates whose energy lies within some specified range.

Let us also calculate the dispersion $\overline{(\Delta E)^2}$ of the energy:

$$\overline{(\Delta E)^2}=\overline{(E-\overline{E})^2}= \over... ...-2E\overline{E}+\overline{E}^2}= \overline{E^2}-\overline{E}^2$$ (19)

We have already computed $\overline{E}$. We need now to compute $\overline{E^2}$:

$$\overline{E^2}=\frac{\sum_rE_r^2 e^{-\beta E_r}}{\sum_r e^{-\beta E_r}}$$ (20)

But

$$\sum_rE_r^2 e^{-\beta E_r}=-\frac{\partial}{\partial \beta} ... ...l}{\partial \beta}\right)^2 \left(\sum_r e^{-\beta E_r}\right)$$ (21)

And from the definition of the partition function $Z$

$$\overline{E^2}=\frac{1}{Z}\frac{\partial^2 Z}{\partial \beta^2}$$ (22)

This can be rewritten as

$$\overline{E^2}=\frac{\partial}{\partial \beta}\left( \frac{1... ...2= -\frac{\partial\overline{E}}{\partial \beta}+\overline{E}^2$$ (23)

Finally we obtain

$$\overline{(\Delta E)^2}=\overline{E^2}-\overline{E}^2= -\frac{\partial\overline{E}}{\partial \beta}$$ (24)

or

$$\overline{(\Delta E)^2}=\frac{\partial^2 \ln Z}{\partial \beta^2}$$ (25)

We can also use $Z$ to generate the mean generalized force $\overline{X}$. Suppose now that we change some macroscopic parameter $x$. Then the energy changes by the amount

$$dE_r=\frac{\partial E_r}{\partial x}dx$$ (26)

and the macroscopic work done by the system is

$$dW=\overline{X}dx=-\overline{\frac{\partial E_r}{\partial x}... ...r\left(-\frac{\partial E_r}{\partial x}\right)e^{-\beta E_r}dx$$ (27)

Now note that in the numerator

$$\sum_r\frac{\partial E_r}{\partial x}e^{-\beta E_r}=-\frac{1... ...eta E_r}\right)= -\frac{1}{\beta}\frac{\partial Z}{\partial x}$$ (28)

Substituting in (27), we obtain

$$dW=\frac{1}{\beta Z}\frac{\partial Z}{\partial x}dx=\frac{1}{\beta} \frac{\partial \ln Z}{\partial x}dx$$ (29)

Recall that

$$dW=\overline{X}dx$$ (30)

where $\overline{X}$ is the generalized force associated with the parameter $x$:

$$\overline{X}\equiv-\overline{\frac{\partial E_r}{\partial x}}$$ (31)

Thus, comparing (29) and (30) leads to

$$\overline{X}=\frac{1}{\beta}\frac{\partial \ln Z}{\partial x}$$ (32)

If $x$ is the volume, then $\overline{X}$ is the pressure $p$:

$$\overline{p}=\frac{1}{\beta}\frac{\partial\ln Z}{\partial V}$$ (33)

Now let's derive a relation between $S$ and $Z$. Note that $Z$ is a function of both $\beta$ and $x$. Thus

$$d\ln Z = \frac{\partial \ln Z}{\partial x}dx + \frac{\partial \ln Z}{\partial\beta}d\beta$$

$$= \beta\overline{X}dx-\overline{E} d\beta$$

$$= \beta dW-d(\overline{E}\beta)+\beta d\overline{E}$$ (34)

or

$$d(\ln Z+\beta \overline{E})=\beta(dW+d\overline{E})=\beta dQ$$ (35)

But since

$$dS=\frac{dQ}{T}$$ (36)

we obtain

$$S=k_B(\ln Z +\beta\overline{E})$$ (37)

or

$$TS=k_BT\ln Z+\overline{E}$$ (38)

or

$$\overline{E}-TS=-k_BT\ln Z$$ (39)

Recall that in thermodynamics $F=\overline{E}-TS$ where $F$ is the Helmholtz free energy. Hence

$$F=-k_BT\ln Z$$ (40)

or

$$Z=e^{-\beta F}$$ (41)

This equation forms the bridge between the canonical ensemble of statistical mechanics and thermodynamics. We can use it to relate the microscopics of the system to the macroscopic parameters that we deal with in thermodynamics.

Notice that since

$$F=-k_BT\ln Z=-\frac{1}{\beta}\ln Z$$ (42)

we can write the mean pressure as

$$\overline{p}=\frac{1}{\beta}\frac{\partial \ln Z}{\partial V}= -\left.\frac{\partial F}{\partial V}\right\vert _T$$ (43)

We obtained this previously using

$$dF=-SdT-pdV$$ (44)

We will relate one final quantity to the partition function: the specific heat at constant volume. Recall that

$$C_y=\left(\frac{dQ}{dT}\right)_{y}=T\left.\frac{\partial S}{\partial T} \right\vert _y$$ (45)

Let $y=V$, then at constant volume

$$dE=dQ-dW=dQ$$ (46)

since $dW=pdV=0$. Thus

$$C_V=\left.\frac{\partial \overline{E}}{\partial T}\right\ver... ...beta^2\frac{\partial\overline{E}}{\partial\beta}\right\vert _V$$ (47)

But

$$\overline{E}=-\frac{\partial \ln Z}{\partial\beta}$$ (48)

Therefore

$$C_V=k_B\beta^2\frac{\partial^2\ln Z}{\partial\beta^2}= k_B\beta^2\overline{(\Delta E)^2}$$ (49)

Notice that the specific heat is related to the fluctuations in the internal energy or, equivalently, to the width of the distribution of $E$. In a numerical simulation, one way to calculate the specific heat is to calculate $\overline{(\Delta E)^2}$. We now see that the partition function contains the information about the system. Most quantities of interest are obtained from the appropriate derivatives of $Z$. The real task in statistical mechanics is to calculate the partition function. Once that is done, all that remains is differentiation.

We can also relate the specific heat to the Helmholtz free energy:

$$F=E-TS$$ (50)

Recall that

$$dF=-SdT-pdV$$ (51)

implies that

$$S=-\left(\frac{\partial F}{\partial T}\right)_V$$ (52)

We got this when we derived $F=F(T,V)$ using a Legendre transformation. We can obtain the specific heat $C_V$ using

$$C_V=T\left.\frac{\partial S}{\partial T}\right\vert _V=-T\left. \frac{\partial^2 F}{\partial T^2}\right\vert _V$$ (53)

This is equivalent to eq. (49).

Grand Canonical Ensemble

Suppose that the system under consideration is in contact with both a particle and energy reservoir. In this case both energy and particle number can be exchanged with the reservoir. In this situation neither the total energy nor the particle number of the system is constant. Two examples of such systems are a liter of air within a larger volume of air, and a 1 cm$^3$ sample of copper within a larger block of copper. For mathematical reasons quantum mechanical systems are most easily treated when in contact with both a heat and particle number reservoir.

Assume that system A can exchange both energy and particles with system A$^{\prime}$. Assume

$$E+E^{\prime} = E^o$$

$$N+N^{\prime} = N^o$$ (54)

Let $\Omega^{\prime}(E^{\prime},N^{\prime})$ be the number of microstates accessible to the reservoir A$^{\prime}$ when it has energy $E^{\prime}$ and contains $N^{\prime}$ particles. The probability $P_r$ of finding A in the microstate $r$ is

$$P_r=C^{\prime}\Omega^{\prime}(E^o-E_r,N^o-N_r)$$ (55)

where $C^{\prime}$ is a constant. Since both $E_r\ll E^o$ and $N_r\ll N^o$,

$$\ln\Omega^{\prime}(E^o-E_r,N^o-N_r)=\ln\Omega^{\prime}(E^o,N... ...Omega^{\prime}}{\partial N^{\prime}}\right\vert _{N^o}N_r +...$$ (56)

Let

$$\beta\equiv\frac{\partial\ln\Omega^{\prime}}{\partial E^{\prime}}$$ (57)

and

$$-\beta\mu\equiv\frac{\partial\ln\Omega^{\prime}}{\partial N^{\prime}}$$ (58)

where $\mu$ is called the chemical potential. Note that both $T$ and $\mu$ are properties of the reservoir and not the system A. If we use the chain rule, then

$$-\beta\mu\equiv\frac{\partial\ln\Omega^{\prime}}{\partial N^... ...prime}} =\beta \frac{\partial E^{\prime}}{\partial N^{\prime}}$$ (59)

This implies that

$$\mu=-\frac{\partial E^{\prime}}{\partial N^{\prime}}$$ (60)

This is consistent with the statement that the chemical potential is the energy required to add a particle or the difference in energy between having $N^{\prime}$ and $N^{\prime}+1$ particles. One way to think about chemical potential is in terms of energy levels of 2 pieces of metal. If the two pieces have different numbers of electrons, when they are put into contact, electrons will flow from one to the other because electrons in a higher energy level in one metal can lower their energy by going to a lower level in the other metal. This flow continues until the electrons are filled up to the same level. This ``level'' is the chemical potential.

=3.0 true in

Back to (56):

$$\Omega^{\prime}(E^o-E_r,N^o-N_r)=\Omega^{\prime}(E^o,N^o) e^{-\beta(E_r-\mu N_r)}$$ (61)

and

$$P_r=Ce^{-\beta(E_r-\mu N_r)}$$ (62)

where

$$C^{-1}=\sum_r e^{-\beta(E_r-\mu N_r)}$$ (63)

It then follows that

$$\overline{E}=\frac{\sum_r E_r e^{-\beta(E_r-\mu N_r)}} {\sum_r e^{-\beta(E_r-\mu N_r)}}$$ (64)

and

$$\overline{N}=\frac{\sum_r N_r e^{-\beta(E_r-\mu N_r)}} {\sum_r e^{-\beta(E_r-\mu N_r)}}$$ (65)

Let

$${\cal{Z}}=\sum_r e^{-\beta(E_r-\mu N_r)}$$ (66)

Then

$$\overline{N}=\frac{1}{\beta}\frac{\partial\ln\cal{Z}} {\partial \mu}$$ (67)

Also

$$\frac{\partial\cal{Z}}{\partial \beta}=\sum_r(-E_r+\mu N_r) e^{-\beta(E_r-\mu N_r)}$$ (68)

or

$$\frac{1}{\cal{Z}}\frac{\partial\cal{Z}}{\partial\beta}= \overline{-E+\mu N}=-\overline{E}+\mu\overline{N}$$ (69)

or

$$\overline{E}=\mu\overline{N}-\frac{\partial}{\partial \beta}\ln\cal{Z}$$ (70)

The function $\cal{Z}$ is called the grand partition function. It is this function which is of primary importance in the grand canonical ensemble. We will return to a consideration of the grand canonical partition function when we begin our study of quantum statistical mechanics.

Before we begin a discussion of the applications of these basic concepts, two useful remarks need to be made. The first is the definition of the partition function within classical mechanics. In clasical mechanics, the sum over microstates is replaced by an integral over phase space. That is

$$Z=\int\frac{dq_1...dq_fdp_1...dp_f}{h_o^f}e^{-\beta E(q_1 ... q_fp_1...p_f)}$$ (71)

A second remark concerns the partition function of two independent systems. Let A and B be two independent systems both in contact with the same reservoir A$^{\prime}$. Let us label the microstates of system A by $r$ and the microstates of system B by $s$. We will assume that the total energy $E_{rs}$ of system A in microstate $r$ and system B in microstate $s$ is

$$E_{rs}=E_r^{A}+E_{s}^{B}$$ (72)

The partition function of the combined system A plus B is

$$Z = \sum_{r,s} e^{-\beta E_{rs}}$$

$$= \sum_{r,s}e^{-\beta(E_r+E_s)}$$

$$= \sum_{r}e^{-\beta E_r}\sum_{s}e^{-\beta E_s}$$

$$= Z_AZ_B$$ (73)

Thus the partition function of two independent systems is just the product of the two independent partition functions. The only assumption has been that the energy of the total system can be expressed as the sum of the energies of the two individual independent systems. Notice that this means we can add free energies:

$$F=-k_BT\ln Z=-k_BT\ln(Z_AZ_B)=-k_BT\ln Z_A-k_BT\ln Z_B=F_A+F_B$$ (74)

The generalization to more than two systems is obvious. Assume we have $N$ identical but independent systems. If $\xi$ is the partition function of one system, then the total partition function of $N$ systems is

$$Z=\xi^{N}$$ (75)

We will find that quantum mechanics will lead to a correction to this equation under certain conditions.