Ensembles

from Statistical Physics using Mathematica
© James J. Kelly, 1996-2002

Microcanonical ensemble

Definition

The simplest and most fundamental ensemble is constructed as the set of isolated systems, each prepared in a unique stationary quantum state, such that each and every state within the energy range {E, E + δ E} is represented exactly once.  The stipulations that each member of the ensemble is prepared in a stationary state and that the systems are isolated from all external interactions imply that the state of each system is constant and the ensemble is stationary.  Since each microstate within the specified energy range is represented once and only once, the statistical weight assigned to each member of the ensemble is simply the inverse of the total number of states available.  Therefore, the probability (Subscript[P, i]) of selecting a particular state i at random from a microcanonical ensemble containing Γ[E, δ E] states is simply

\(TextForm\`P\_i = 
      1\/Γ[E, δ E]\ \ \ \ for\ E ≤ E\_i ≤ 
        E + δ E\)
(Subscript[P, i]) =  
      0                      otherwise

Ensemble averages for property \(TraditionalForm\`x\&^\) are then computed as unweighted averages over the values (Subscript[x, i]) for each of the Γ members of the ensemble, according to

\(TextForm\`⟨\(x\&^\)⟩\  = \ \(∑\+\(i\  ∈ \ Γ\)\(P\_i\) x\_i\  = 
        1\/Γ\ \(∑\+\(i\  ∈ \ Γ\)x\_i\)\)\)

The thermodynamic internal energy U is the ensemble average of \(TraditionalForm\`\(H\&^\)\), such that

\(TextForm\`U\  = \ \(⟨\(H\&^\)⟩\  = \ \(∑\+\(i\  ∈ \ Γ\)\(P\_i\) E\_i\  = 
          1\/Γ\ \(∑\+\(i\  ∈ \ Γ\)E\_i\)\)\)\)

Similarly, if the volume or particle number vary within the ensemble, the thermodynamic volume and particle number are also interpreted as ensemble averages.

Entropy in a microcanonical ensemble is obtained directly from the multiplicity function Γ[E, δ E] = 
        g[E] δ E according to

S  =  (Subscript[k, B]) 
          ln Γ  ≈  (Subscript[k, B]) 
          ln g

where the energy interval is usually sufficiently small that for systems with large multiplicity there is no practical (logarithmic) difference between the total number of states Γ within the interval and the density of states g[E] at the central energy of the interval.  Similarly, if this interval is small, there is no practical difference between E and U, so that g[E] -> g[U].  Thus, the fundamental thermodynamic relation for the microcanonical ensemble specifies the dependence of entropy upon internal energy and any other thermodynamic variables needed to specify the ensemble.  All other thermodynamic properties of the system can be obtained from this fundamental relation.  For example, the temperature can be obtained using

\(TextForm\`1\/T = \(\(∂S\/∂U\)\ \ \ \ ⟹\ \ \ \ β\  = \ \(1\/\(\(k\_B\) T\)\  = \ \(∂ln\ g[U]\)\/∂U\)\)\)

where other variables are held constant.  In the microcanonical ensemble temperature measures the energy dependence of the multiplicity function for isolated systems.  For isolated systems, you specify the mean energy and then the internal dynamics decide the temperature.

The next few sections provide examples of the application of the microconical ensemble to prototypical systems which illustrate some important features of thermostatistics.  Here we summarize and discuss the results, leaving technical details and more thorough development to separate notebooks.

Example: binary systems

Consider a system of N weakly interacting elements for which each can occupy only two states.  For example, if we neglect kinetic energies and mutual interactions, a system of N spin (1/2) atoms or nuclei subject to an external magnetic field can be approximated as an ideal two-level system.  Suppose that (Subscript[N, 1]) elements are found in the state with energy (Subscript[ε, 1]) = -ε and (Subscript[N, 2]) in the state with energy (Subscript[ε, 2]) = +ε.  It is convenient to define an order parameter -1 ≤ x ≤ 1, or alignment variable, to represent the excess population of the lower level

\(TextForm\`		N\_1 = \(\(N\/2\) \((1 + 
              x)\)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ N = N\_1 + N\_2\)\)
\(TextForm\`		N\_2 = \(\(N\/2\) \((1 - 
              x)\)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ x\  = \ \((N\_1 - N\_2)\)/N\)\)

such that the energy becomes

\(TextForm\`U\  = \ \(\(N\_1\) ε\_1\  + \ \(N\_2\) ε\_2\  = \(-N\)\ x\ ε\)\)

The number of states with (Subscript[N, 1]) elements in level (Subscript[ε, 1]) and N - (Subscript[N, 1]) elements in level (Subscript[ε, 2]) is given by the binomial coefficient

g[x] = (({{N}, {(Subscript[N, 1])}})) = \(\(N!\)\/\(\(N\_1!\) \(N\_2!\)\)\  = \ \(N!\)\/\(\(\((\(N\/2\) \((1 + x)\))\)!\) \(\((\(N\/2\) \((1 - x)\))\)!\)\)\)

The entropy S[x] is proportional to the logarithm of the multiplicity, which with the aid of Stirling's formula and straightforward algebra becomes

S = 
      N (Subscript[k, B]) ( ((1 + x)/2) 
              Log[(2/(1 + x))]  +  ((1 - x)/2) 
              Log[(2/(1 - x))]  )

The temperature is determined by the thermodynamic relation

\(TextForm\`1\/T = \(∂S\/∂U = \ \(\(-\(1\/\(N\ ε\)\)\) \(∂S\/∂x\)\ \ \ \ ⟹\ \ \ \ \ \ \(ε\/\(\(k\_B\) 
                    T\)\) = \(1\/2\) 
            Log[\(1 + x\)\/\(1 - x\)]\)\)\)

Solving for the order parameter, we find

\(TextForm\`x = \(Tanh[β\ ε]\ \ \ ⟹\ \ \ U\  = \ \(-N\)\ ε\ Tanh[β\ ε]\)\)

where β ≡ 1/(Subscript[k, B])  T.  After some tedious algebra, we can express entropy and free energy in terms of temperature as

\(TextForm\`S = \(N\ \(k\_B\) \((\ 
              Log[2  Cosh[β\ ε]]\  - \ β\ ε\ Tanh[β\ ε]\ \ )\)\ \ ⟹\ \ \ F = \(-N\)\ \(k\_B\) 
          T\ \ Log[2  Cosh[β\ ε]]\)\)

Finally, the heat capacity becomes

\(TextForm\`C = \(∂U\/∂T\  = \ 
        N\ \(k\_B\) \((\ β\ ε\ Sech[β\ ε]\ )\)\^2\)\)

where we assume that any external parameters that govern ε, such as magnetic field, are held fixed.

The figure below shows the dependence of entropy upon the order parameter.  When x -> +1 all elements are found in the lower energy level and the system has its lowest possible energy.  The entropy vanishes when the system is completely ordered and the temperature approaches zero, \(TraditionalForm\`T -> \(0\^+\)\).  More states become available as the energy and temperature of the system increases.  As T-> = ∞ both upper and lower states are populated equally and the entropy approaches its maximum possible value, (Subscript[S, max]) = N (Subscript[k, B]) ln 2.  The temperature becomes infinite at U = 0 because the slope of S[U] vanishes, but is positive for slightly negative U and appears to be negative for slightly positive U.  As the energy increases further an increasing fraction of the elements are found in the upper level and the entropy decreases as the system become increasingly ordered.  In this region the temperature appears to be negative because the slope of S[U] is negative.  Finally, the entropy vanishes as the energy approaches its maximum possible value, corresponding to \(TraditionalForm\`T -> \(0\^-\)\).

The concept of negative temperature can be disconcerting, at first, but it is a characteristic feature of isolated systems with bounded ranges of energy and entropy.  Any system for which the entropy is a convex function of energy, as shown above, can find itself with entropy decreasing as the upper limit of the energy range is approached.  If the slope of S[U] is negative, so is the temperature.  However, one would naturally describe increasing energy as heating, such that states with negative temperature are actually hotter than states with positive temperature.  The figure below shows the dependence of temperature on internal energy: \(TraditionalForm\`T = \(0\^+\)\) is cold, T increases as the system is heated, and T -> +∞ as both states are populated equally.  As the system is heated further the temperature increases from +∞ to -∞, which are infinitesimally close in energy, and then becomes less negative until the hottest temperature is reached at \(TraditionalForm\`T = \(0\^-\)\).

The classic example of negative temperature is provided by nuclear spin systems which interact so weakly with the surrounding crystal lattice that macrostates with nuclear magnetization opposed to the external magnetic field can be maintained in constrained or partial equilibrium for a long time.  This phenomenon was first demonstrated by Purcell and Pound.  The nuclear spins were aligned parallel to a strong magnetic field.  The field was then reversed so rapidly that the spins remained in their original orientation and found themselves aligned antiparallel with the magnetic field.  Hence, the nuclear spin system is characterized by negative temperature.  The spin system then gradually gives up its excess internal energy and returns to a state aligned with the external field with a relaxation time of the order of 5 minutes.  Although the macrostate changes and is not truly in equilibrium at negative temperature, the relaxation time is so long compared with atomic time scales that it can be considered to be a quasistatic process that is always sufficiently near equilibrium to apply the concepts and methods of equilibrium statistical mechanics.

A bounded system with greater population of the upper energy level is described as having a population inversion.  Such systems are not normally in a true equilibrium because it is not possible in practice to isolate a system from its environment.  The temperature of the environment will always be positive because the energy of any statistically normal system with kinetic contributions is unbounded.  Recognizing that the any system with negative temperature is hotter than any system with positive temperature, heat will flow from the system with negative temperature until it eventually reaches thermal equilibrium at positive temperature. Nevertheless, if the relaxation time is sufficiently long, such as some nuclear spin systems, the system can be maintained in partial or constrained equilibrium long enough to observe negative temperature.  Other systems, such as lasers, require active pumping to maintain the population inversion.

For a system whose energy is limited to a finite range, the entropy will generally be small at the extreme values of the internal energy and maximal somewhere in between, as seen above for the two-level system.  The temperature, \(TraditionalForm\`T = \((∂\ S\/∂\ U)\)\^\(-1\)\), will then be positive for U below the entropy peak, negative above, and infinite at the peak where ∂ (S/∂) U = 0.  The figure below illustrates the dependence of temperature upon internal energy and shows that there is a singularity at U = 0.  The difference in energy between states with very large negative temperature and states with very large positive temperature is actually very small.  Therefore, the heat capacity must vanish at the entropy peak because an infinitesimal change in energy produces an infinite change in temperature.

The principal thermodynamic functions for a binary system are plotted below as functions of a dimensionless reduced temperature variable τ ≡ (Subscript[k, B]) T/ε defined as the ratio between the thermal energy (Subscript[k, B]) T and the characteristic excitation energy ε.  The discontinuity in internal and free energies at τ = 0 is associated with the zero entropy for fully ordered systems in which all elements are found either in the lower state for \(TraditionalForm\`τ -> \(0\^+\)\) or the upper state for \(TraditionalForm\`τ -> \(0\^-\)\).  The heat capacity vanishes at τ = 0 because a fully ordered system cannot absorb heat without a finite change of temperature.  Similarly, the capacity also vanishes at T = ±∞ because a system with maximum entropy has \(TraditionalForm\`∂S\/∂T = \(0\ ⟹\ C = 0\)\).  Therefore, the heat capacity must exhibit a maximum for intermediate temperatures.  This characteristic feature of systems with bounded energy is called a Schottky anomaly; it is anomalous because the heat capacity for normal systems with unlimited capacity to absorb energy increases monotonically with temperature.

Finally, it is useful to examine the limiting behaviors of these functions.  For high temperatures we find

\(TextForm\`			S\/\(N\ k\_B\) ∼ \ 
      ln\ 2\  - \ y\^2\/2\  + \ 
        y\^4\/4\  + \ ⋯\)
\(TextForm\`			U\/\(N\ ε\) ∼ \ \(-y\) + 
        y\^2\/3\  + \ ⋯\)
\(TextForm\`			C\/\(N\ k\_B\) ∼ \ 
      y\^2 - y\^4\)

while at low temperatures

\(TextForm\`			S\/\(N\ k\_B\) ≈ \(e\^\(\(-2\) y\)\) \((1 + 2  y)\)\)
\(TextForm\`			U\/\(N\ ε\) ≈ \ \(-\((1 - 
            2  e\^\(\(-2\) y\))\)\)\)
\(TextForm\`			C\/\(N\ k\_B\) ≈ \ 
      4 \( y\^2\) e\^\(\(-2\) y\)\)

where y = (τ^(-1)) is a dimensionless reduced energy variable defined as the reciprocal of the reduced temperature.  The exponential factors obtained for low temperatures are characteristic of systems with a gap in their excitation spectrum.  At very low temperature most elements are found in the lowest energy level and the population of the upper energy level is

((Subscript[N, 2])/(Subscript[N, 1])) ≈ (e^(-2 y)) = 
      Exp[-(Δ/((Subscript[k, B]) T))]

where \(TraditionalForm\`Δ = \(ε\_2 - ε\_1 = 2  ε\)\) is the energy gap between the two levels.  Hence, the heat capacity is small because the entropy and the change in entropy with temperature are small.  Systems with a continuous excitation spectrum normally exhibit power-law behavior at low temperature, but the finite gap for the binary system produces exponential suppression of the low-temperature heat capacity.

Algebraic details and more thorough presentations of the thermodynamics of binary systems can be found in spin-half.nb using the microcanonical ensemble and  thermo2.nb using the canonical ensemble.

Example: harmonic oscillators

Consider a system of harmonic oscillators and assume that interactions between these oscillators can be neglected.  For example, electromagnetic radiation in a cavity, vibrational excitations of molecules, or the vibrations of atoms about lattice sites in a crystal may be analyzed in terms of independent normal modes of oscillation.  Each individual oscillator has an energy spectrum \(TraditionalForm\`ε\_n = ℏ \(ω(n + 1\/2)\)\) consisting of an infinite sequence of equally spaced levels where n is the total number of quanta, ℏ ω is the fundamental quantum of energy for the system, and the zero-point energy is ℏ ω/2.

For example, consider a single particle in a 3-dimensional harmonic well.  The total number of quanta is then n = (Subscript[n, x]) + (Subscript[n, y]) + (Subscript[n, z]) where each of the three spatial dimensions can be treated as an independent oscillator.  The degeneracy of each single-particle energy level is equal to the total number of ways that n quanta can be distributed among the three independent modes.  There are n + 1 possible values of (Subscript[n, x]) between 0 and n.  For each of these values, we can choose (Subscript[n, y]) to be anywhere between 0 and n - (Subscript[n, x]).  The number of quanta along the z-axis is then determined.  Hence, the degeneracy of a single-particle level with n quanta is given by the sum

\(TextForm\`g[
        n] = \(∑\+\(n\_x = 0\)\%n\((n - n\_x + 
              1)\)\  = \ \(∑\+\(k = 1\)\%\(n + 1\)k\  = \ \(1\/2\) \((n + 1)\) \((n + 2)\)\)\)\)

whose value can be demonstrated by induction.

The degeneracy for a system of many independent oscillators can now be obtained by extending this argument to f independent degrees of freedom, where f is the total number of oscillator modes rather than simply the total number of particles.  Let n represent the total number of quanta, so that the internal energy of the system is U = ℏ ω(
            n + (f/2)).  The degeneracy is simply the number of distinct ways that n indistinguishable objects (quanta) can be distributed among f distinguishable boxes (vibrational modes).  Suppose that the boxes are represented by vertical lines marking their boundaries and that the objects are represented by circles, as sketched below for a particular configuration.

•• | •••• || ••• | ••••• || •

The outermost walls need not be included.  The number of combinations of n objects and f - 1 interior partitions is simply (f + n - 1)!.  However, because the n! permutations of the balls or the (f - 1)! permutations of the partitions among themselves leave the system invariant, the degeneracy is simply the binomial coefficient

g[n, f] =(({{f + n - 1}, {n}})) = (((f + n - 1)!)/(n! (f - 1)!))

The earlier result for a single three-dimensional oscillator is recovered by choosing f -> 3.

In the limit of large n and large f, we can employ Stirling's approximation to obtain

ln g  ≈  (f + n) ln[f + n]  -  
          n ln n  =  
      f ( (1 + x) ln[1 + x]  -  x ln x )

where x = n/f is the average excitation per oscillator.  The entropy can now be expressed as

S  =  (Subscript[k, B]) 
        f ( (1 + x) ln[1 + x]  -  x ln x )

where x + (1/2) = 
        U/f ℏ ω.  Evaluating the temperature, we find

\(TextForm\`1\/\(\(k\_B\) T\) = \(\(\(∂\ 
                    ln\ g\)\/∂U\)\ \ \ ⟹\ \ \ \ f\ y\  = \ \(\(∂\ ln\ g\)\/∂x\  = \ \(f\ ln\ \(\(1 + 
                      x\)\/x\)\ \ \ \ ⟹\ \ \ \ x\  = \ 
            1\/\(e\^y - 1\)\)\)\)\)

where y = ((ℏ ω)/((Subscript[k, B]) T)) is a dimensionless variable that represents the ratio between oscillator and thermal energies.  Therefore, we obtain the thermal equation of state

U  =  f ℏ ω ( 
            (1/((e^(ℏω/(Subscript[k, B])  T)) - 1))  +  
              (1/2))  = (( f ℏ ω)/2) 
          Coth[y]

and heat capacity

\(TextForm\`C\_V\/\(f\ k\_B\) = \ \(\(\(y\^2\) e\^y\)\/\((e\^y - 1)\)\^2\ \ \ \ where\ \ \ \ y = \(ℏ\ ω\)\/\(\(k\_B\) T\)\)\)

It is useful to define a dimensionless reduced temperature as τ = (y^(-1)) = (Subscript[k, B]) 
            T/ℏ ω.  At high temperature, we can expand these functions is powers of the small variable y, to find

\(TextForm\`		U\  ≈ \ 
      f\ \(k\_B\) 
        T \((1 + \(1\/12\) \((\(ℏ\ ω\)\/\(\(k\_B\) T\))\)\^2\  + \ ⋯)\)\)
\(TextForm\`		C\_V\  ≈ \ 
      f\ \(k\_B\) \((1 - \(1\/12\) \((\(ℏ\ ω\)\/\(\(k\_B\) T\))\)\^2 + \ ⋯)\)\)

Thus, at high temperature the heat capacity  per oscillator approaches the limiting value (Subscript[k, B]).  At low temperature, where y is large, we isolate the exponential factors to obtain

\(TextForm\`		U\  ≈ \ \(\(\(\ \)\(f\ ℏ\ ω\)\)\/2\) \((\ 
          1\  + \ e\^\(-y\))\)\)
\(TextForm\`		C\_V\  ≈ \ 
      f\ \(k\_B\) \(y\^2\) e\^\(-y\)\)

Notice that at low temperature the heat capacity resembles that of the binary system because only the first excited state matters when the thermal energy is small compared with the excitation energy.  Under those circumstances the thermodynamics is dominated by the gap in the excitation spectrum.  However, unlike the binary system, the internal energy is proportional to temperature and the heat capacity approaches a nonzero constant at high temperature because the energy spectrum is not bounded.  Thus, the oscillator system is considered statistically normal while the binary system is not.

Einstein's model of the heat capacity of a crystal containing N atoms in a lattice treated the 3  N normal modes of vibration as independent harmonic oscillators with a single average frequency (Subscript[ω, E]).  If we define a characteristic temperature (Subscript[T, E]) = ℏ (Subscript[ω, E])/(Subscript[k, B]), the reduced temperature becomes τ = T/(Subscript[T, E]).  The figure below displays the dependence of the heat capacity upon reduced temperature.  At high temperatures, τ ≫ 1, one finds the classical heat capacity (Subscript[C, V]) -> 3 N (Subscript[k, B]) expected for equipartition among 3  N coordinates and 3 N momenta.  However, at low temperatures, τ < 1, the heat capacity is much smaller quantum mechanically than classically because the finite energy required to excite a quantized oscillator freezes out states that would participate in a classical theory that permits arbitrarily small excitations.  Although this model is simplistic, the explanation of reduced heat capacity at low temperature in terms of quantization of energy provided important support for the early quantum theory.

We will return to this and more refined models shortly.  Further information on these systems can be found in the notebooks hotherm.nb, debye.nb, and planck.nb.

Canonical ensemble

Thermal interaction with a reservoir

Suppose that a large isolated system is partitioned asymmetrically into two parts, one of which is very much larger than the other, separated by a rigid, impermeable wall which nevertheless conducts heat.  The total energy E of the composite system is conserved.  We seek the probability (Subscript[P, i]) that, in thermal equilibrium, the small system is found in a particular quantum state i with energy (Subscript[ε, i]).  The large system serves as a heat reservoir with total energy (Subscript[E, R]) = E - (Subscript[ε, i]) that is much greater than the energy (Subscript[ε, i]) ≪ (Subscript[E, R]) of the system of interest.  Henceforth, we refer to the large system as the "reservoir" and to the smaller subsystem as simply the "system".

Energy sharing between a reservoir with energy (Subscript[E, R]) = E - (Subscript[ε, i]) and a small system with energy (Subscript[ε, i]) in state i.  Thick walls isolate the composite system while the two subsystems exchange heat through the thin rigid, impermeable, diathermal wall.

Although we have specified the state of the system precisely, the reservoir may occupy any of its states which satisfy the constraint (Subscript[E, R]) + (Subscript[ε, i]) = E upon the total energy.  According to the postulate of equal a priori probabilities, each of these states is found with equal likelihood.  Therefore, (Subscript[P, i]) is proportional to the multiplicity (Subscript[Ω, R]) for the reservoir, such that

(Subscript[P, i]) = 
      C (Subscript[Ω, R])[E - (Subscript[ε, i])]

where C is a normalization constant.  Expanding ln (Subscript[Ω, R]) for ε ≪ E,

\(TextForm\`ln\ Ω\_R[E - ε]\  ≈ \ 
      ln\ Ω\_R[
            E]\  - \ ε\ \(∂ln\ Ω\_R\)\/∂E\)

and identifying the entropy and temperature of the reservoir as

\(TextForm\`S\_R = \(\(k\_B\) 
          ln\ Ω\_R\ \ \ \ \ \ \ \ \ \ \ \ \ \ 1\/T = ∂S\_R\/∂E\)\)

we find

ln (Subscript[P, i]) = -((Subscript[ε, i])/((Subscript[k, B]) T)) +  
        (C^(′ ))

where (C^(′ )) is some other constant.  Therefore, the relative probability that the system occupies a particular quantum state is given by the Boltzmann factor

\(TextForm\`P\_i ∝ \ e\^\(\(-β\)\ ε\_i\)\ \ \ \ \ \ \ \ \ \ \ \ \ β ≡ 
      1\/\(\(k\_B\) T\)\)

To interpret this result, consider two states labelled 1 and 2 with energies (Subscript[ε, 1])and (Subscript[ε, 2]), respectively.  The ratio of probabilities for finding the system in either of these states, in thermal equilibrium with a reservoir at temperature T, is then

((Subscript[P, 1])/(Subscript[P, 2])) =  
      Exp[-Δ(ε/((Subscript[k, B]) 
                T))]

where \(TraditionalForm\`Δε = ε\_1 - ε\_2\) is the energy difference between the two states.  The state with higher energy occurs with smaller probability because the number of states available to the reservoir, and hence the entropy of the reservoir, is smaller if less energy is available to it.  This effect is relatively small if the temperature is large enough so that the typical thermal energy (Subscript[k, B]) T is much larger than the energy differences among states of the smaller system.  Under these circumstances all states of the system are populated with essentially equal frequency,

\(TextForm\`\(k\_B\) 
        T ≫ Δε\ \ \ ⟹\ \ \ P\_1 ≈ P\_2\)

corresponding to a highly disordered state of high entropy.  On the other hand, if the thermal energy is comparable to or smaller than the spacing between energy levels of the system, the probability that a level with energy ε is populated decreases exponentially with energy.  Under these circumstances, states of higher energy are effectively frozen out:

\(TextForm\`\(k\_B\) 
        T ≲ Δε\ \ \ ⟹\ \ \ P\_1 ≪ P\_2\)

With fewer quantum states populated with appreciable frequency, the system exhibits a higher degree of order and possesses lower entropy.

Therefore, the canonical probability distribution

\(TextForm\`P\_i = \(\(Z\^\(-1\)\) e\^\(\(-β\)\ ε\_i\)\ \ \ \ \ \ \ \ \ \ β = \(1\/\(\(k\_B\) T\)\ \ \ \ \ \ \ \ \ Z = ∑\+i e\^\(\(-β\)\ ε\_i\)\)\)\)

describes the population of states for a small system in thermal equilibrium with a much larger reservoir of heat.  The reservoir must be large enough so that its macroscopic properties are not affected by fluctuations in the energy drawn by the smaller system.  The reservoir then defines the temperature of the smaller system precisely even if its energy fluctuations are relatively large (compared with its mean energy).  The relative probability for each quantum state of the system is described by the Boltzmann factor \(TraditionalForm\`e\^\(\(-β\)\ ε\)\) and the absolute probability is obtained by normalizing the distribution to unity.  The normalization factor is determined by the partition function Z, where the sum extends over all possible states of the system.  The Boltzmann factor plays a central role throughout the theory of statistical mechanics and, hence, is of fundamental significance.

It is important to remember that (Subscript[P, i]) is the probability for a unique quantum state.  The probability P[(Subscript[ε, i])] for finding any state with energy (Subscript[ε, i]) and degeneracy g[(Subscript[ε, i])] is then (Subscript[P, i]) g[(Subscript[ε, i])], such that

\(TextForm\`P[ε\_1]\/P[ε\_2] = 
      g[ε\_1]\/g[ε\_2]\ Exp[\(-\(Δε\/\(\(k\_B\) T\)\)\)]\)

is the ratio of probabilities for two energy levels.  Similarly, it is important to remember that the partition function is defined by a sum over microstates rather than over energy levels.  If an energy level (Subscript[ε, i]) has degeneracy (Subscript[g, i]), it must be counted not once but (Subscript[g, i]) times.  If in a particular application it is more convenient to sum over energy levels instead of states, then each level must be assigned a statistical weight equal to its degeneracy.  The definition of the partition function then becomes

\(TextForm\`Z = ∑\+i\( 
              g\_i\) e\^\(\(-β\)\ ε\_i\)\)

where the summation index now enumerates levels.  Finally, if the spacing between energy levels becomes sufficiently small and the average number of states within a suitable interval dε is a smooth function of energy, we can replace the sum by an integral , such that

\(TextForm\`Z = ∫dε\ g[ε] e\^\(\(-β\)\ ε\)\)

where g[ε] is the density of states defined as the total number of states between ε and \(TraditionalForm\`ε + dε\) according to

g[ε] dε = (∑_(\(ε ≤ ε\_i ≤ ε + dε\)))(Subscript[g, i])

However, we must remember that if a single state or small group of states becomes important, it may be necessary to return to explicit summation because integration with respect to a smooth density of states may miss important effects arising from details of the spectrum of quantum states.  The phenomenon of Bose condensation is the most famous example of this type.

It is also important to recognize that the derivation of the canonical ensemble does not require the system to be of macroscopic size.  In fact, we can apply the Boltzmann factor to analyze the relative probabilities of the states of a single atom which interacts with a reservoir.  The reservoir, of course, must be large enough to define a statistically significant temperature.  However, it need not be a material system — it could, for example, be a thermal distribution of electromagnetic radiation which bathes an atom, taken to be the system.  We then imagine that there exists a statistically large ensemble of macroscopically identical systems interacting with similar environments.  The ensemble then represents a thermodynamic system subject to statistical analysis.  The properties of such an ensemble may exhibit large dispersions, but can be analyzed using the canonical probability distribution.  The members of the ensemble need not be large enough themselves to possess narrow distributions of properties.  Thus, statistical mechanics is more general than thermodynamics, which relies on extensive potentials.  Those potentials need not be strictly extensive for small systems.

Maximizing disorder given fixed mean energy

We wish to construct an ensemble consisting of many macroscopically identical isolated systems for which the mean energy of the ensemble is specified.  The probability distribution {(Subscript[P, i])} should be chosen to maximize disorder subject to the constraints

\(TextForm\`U = ∑\+i\( P\_i\) 
          E\_i\ \ , \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ∑\+i P\_i = 1\)

Thus, using

\(TextForm\`		dS = \(\(-k\_B\) \(∑\+i 
              ln\ \(P\_i\) dP\_i\)\  = \ 
        0\)\)
\(TextForm\`		dU = \(∑\+i\( 
                E\_i\) dP\_i\  = \ 
        0\)\)
\(TextForm\`		∑\+idP\_i = 0\)

and introducing Lagrange multipliers with convenient factors, we find

\(TextForm\`\(\(		\)\(ln\ P\_i\  + \ \(λ\_1\) 
          E\_i\  + \ λ\_2\  = \ \(0\ \ ⟹\ 
          P\_i = Exp[\(-λ\_1\) 
              E\_i - λ\_2]\)\)\)\)

Applying the constraints, we find

\(TextForm\`		λ\_2 = 
      ln \(∑\+i e\^\(\(-λ\_1\) E\_i\)\)\)
\(TextForm\`		U = \(-\(∂λ\_2\/∂λ\_1\)\)\)
\(TextForm\`		S = \(k\_B\) \((\(λ\_1\) 
              U + λ\_2)\)\)

The Lagrange multipliers can be connected to conventional thermodynamic quantities using

\(TextForm\`\(\(		\)\(1\/T = \(\(∂S\/∂U\)\ \ \ ⟹\ \ λ\_1 = 
        1\/\(\(k\_B\) T\) ≡ β\)\)\)\)

Notice that we are holding external parameters, such as volume, fixed so that for this partial derivative the energy levels (Subscript[E, i]) and the normalization (Subscript[λ, 2]) are constant.  Thus, we find that the normalization factor is related to the Helmholtz free energy by

\(TextForm\`\(\(		\)\(λ\_2 = \(\(T\ S\  - \ U\)\/\(\(k\_B\) T\) = \(-β\)\ F\)\)\)\)

Once again defining the canonical partition function Z as the normalization of the probability distribution, the canonical probability distribution which maximizes the disorder within an ensemble of specified mean energy now becomes

\(TextForm\`		P\_i = \(e\^\(\(-β\) E\_i\)\/Z = e\^\(\(-β\) \((E\_i - F)\)\)\)\)
\(TextForm\`		Z\  = \(∑\+i e\^\(\(-β\) E\_i\)\  = \ e\^\(\(-β\) F\)\)\)

Therefore, we have derived the same canonical probability distribution from two rather different perspectives.  The method based upon the thermal interaction between a system (sample) and a much larger reservoir of heat (environment) specifies the temperature through the variation of the entropy of the reservoir with the energy drawn by the sample.  The entropy of the composite system is maximized when all states of the reservoir with the energy determined by the state of the sample are equally likely.  As the sample and the reservoir interact, the state of each changes rapidly.  Hence, the probability for a microstate of the sample represents the fraction of the time a particular sample can be found in that state and is governed by the temperature of the reservoir.  Conversely, the method based upon the entropy of the sample specifies the mean energy for an ensemble of isolated systems and derives the temperature of the sample from the energy dependence of the disorder within that ensemble.  Here the probability for a microstate represents its frequency within an ensemble of stationary states.  This method is closely related to the microcanonical ensemble, but does not require the internal energy to be constrained to an infinitesimal interval.  Both methods give equivalent results because the ergodic hypothesis requires objective and a priori probabilities to be equal.  Both of these methods can be generalized to include other interactions between the sample and its environment.  For example, if we allow the sample and the reservoir to exchange volume through motion of the wall between them, the thermal-interaction method would include the dependence of the multiplicity function for the reservoir upon volume as well as energy.  Alternatively, the information-theory method would need another Lagrange multiplier to constrain the mean volume of the sample.  Both approaches will produce the same probability distribution for this (T, p) ensemble; your preference is an aesthetic choice.

Thermodynamics of canonical ensembles

Canonical ensembles are represented by probability distributions of the form

\(TextForm\`		P\_i = \(\(Z\^\(-1\)\) e\^\(\(-β\)\ E\_i\)\ \ \ \ \ \ ⟺\ \ \ \ \ P[
              E]\  = \ \(Z\^\(-1\)\) 
          g[E]\ e\^\(\(-β\)\ E\)\)\)
\(TextForm\`		Z\  = \(∑\+i e\^\(\(-β\) E\_i\)\ \ \ \ \ \ \ \ ⟺\ \ \ \ \ Z = ∫dE\ g[
              E]\ e\^\(\(-β\)\ E\)\)\)

The ensemble average for any variable x is then

\(TextForm\`⟨\(x\&^\)⟩\  = \ \(∑\+i\( P\_i\) 
            x\_i = \(\(Z\^\(-1\)\) \(∑\+i\( e\^\(\(-β\) E\_i\)\) 
                x\_i\ \ \ \ ⟺\ \ \ ⟨\(x\&^\)⟩\) = \(Z\^\(-1\)\) \(∫dE\ g[E]\ \(e\^\(\(-β\)\ E\)\) 
                x[E]\)\)\)\)

For example, the average internal energy becomes

\(TextForm\`U = \(⟨H\&^⟩\  = \(\(∑\+i\( e\^\(\(-β\) E\_i\)\) E\_i\)\/\(∑\+i e\^\(\(-β\) E\_i\)\)\  = \ \(\(-\(1\/Z\)\) \((∂Z\/∂β)\)\  = \ \(-\(∂\/∂β\)\) ln\ Z\)\)\)\)

Therefore, the partition function determines the thermodynamic properties of the system completely.

It is useful to relate the partition function to the Helmholtz free energy, which is the thermodynamic potential whose natural variables are those specified in the definition of the canonical ensemble, namely temperature and volume.  Recognizing that S = -(Subscript[((∂(F/∂)T)), V]), we identify

F = U - T S  =  
        U + T (Subscript[((∂(F/∂)T)), V])

Using

\(TextForm\`T ∂\/∂T = \ \(-β\)\ ∂\/∂β\)

we can write

\(TextForm\`U = \(F + β \((∂F\/∂β)\)\_V = \(\(∂\/∂β\) \((β\ F)\)\  = \ \(-T\^2\) \(∂\/∂T\) \((F\/T)\)\_V\)\)\)

Thus, comparing expressions for U, we find

\(TextForm\`\(∂\/∂β\) \((β\ F)\)\  = \ \(\(-\(∂\/∂β\)\) 
          ln\ Z\ \ \ ⟹\ \ \ F\  = \ \(-k\_B\) 
            T\ ln\ Z\  - \ T\ S\_0\)\)

where (Subscript[S, 0]) is a constant of integration.

The constant of integration can be determined by examining the behavior of the entropy for temperatures so low that the partition function is dominated by the quantum state of lowest energy, (Subscript[E, 0]).  We then find that

T -> 
      0   ⟹   F ≈ 
        (Subscript[E, 0]) - T (Subscript[S, 0])   ⟹  S ≈ 
        (Subscript[S, 0])

However, the third law of thermodynamics stipulates the entropy approaches zero as the temperature approaches absolute zero.  Microscopically, if we assume that the ground state is nondegenerate, or at least has macroscopically negligible degeneracy, then the entropy must vanish when the temperature is so low that only the ground state is available to the system.  Thus, we conclude that (Subscript[S, 0]) = 0.

Therefore, the thermodynamics of any system can be derived from knowledge of its partition function and the general relationship

\(TextForm\`F = \(\(-k\_B\) 
          T\ ln\ Z\ \ \ ⟹\ \ Z\  = \ e\^\(\(-β\)\ F\)\)\)

We can then compute U by differentiating ln Z with respect to β or by evaluating S from F and using U = F + T S.  Finally, the equation of state p = p[T, V] is most readily determined from the free energy F[T, V] using p = -(Subscript[((∂(F/∂)V)), T]).  These relationships are summarized below.

\(TextForm\`	
    Z = \(e\^\(\(-β\)\ F\)\ \ \ \ \ \ \ \ \ \ ⟹\ \ F = \(-k\_B\) 
          T\ ln\ Z\)\)
\(TextForm\`	
    F = \(U - 
          T\ S\ \ \ \ ⟹\ \ \(dF\)\  = \ \(-S\) dT\  - \ 
          p dV\ \ \((simple\ compressible)\)\)\)
\(TextForm\`	
    S = \(-\((∂F\/∂T)\)\_V\)\)
\(TextForm\`	
    p = \(-\((∂F\/∂V)\)\_T\)\)
\(TextForm\`	
    U\  = \ \(\(-\(∂\/∂β\)\) 
          ln\ Z\  = \ \(\(∂\/∂β\) \((β\ F)\)\  = \ \(\(-T\^2\) \(∂\/∂T\) \((F\/T)\)\_V = 
            F + T\ S\)\)\)\)

Properties of the partition function

The statistical analysis of many systems will be found to benefit from several general properties of the partition function.  First, suppose that the energy of each state is shifted by the same constant δ, such that

\(TextForm\`E\_i -> \(E\_i + δ\ \ \ ⟹\ \ \ Z -> \(e\^\(\(-β\)\ δ\)\) 
          Z\)\)

We then find that the energy and free energy are shifted by the same amount, whereas the entropy and pressure are unaffected.

\(TextForm\`E\_i -> \(E\_i + δ\ \ \ ⟹\ \ \ F -> F + δ\), \ \ U -> U + δ, \ \ S -> 
      S, \ \ p -> p\)

Second, suppose that two parts of a system interact weakly so that their energies are additive.  We then find that the partition function reduces to the product of two separate partition functions, one for each part, and that the free energy reduces to the sum of two separate free energies.  The state of a separable (or factorizable) system can be described by two indices representing the separate states of the two independent systems, such that (Subscript[E, i, j]) = (Subscript[E, i]^((1))) + (Subscript[E, j]^((2))) is the total energy of the product state (i, j) composed of contributions (Subscript[E, i]^((1))) from subsystem (1) and (Subscript[E, j]^((2))) from subsystem (2).  We then find

(Subscript[E, i, j]) = (Subscript[E, i]^((1))) + 
          (Subscript[E, j]^((2)))  ⟹  Z = (Subscript[Z, 1]) 
            (Subscript[Z, 2])   ⟹  F = 
          (Subscript[F, 1]) + (Subscript[F, 2])

where

\(TextForm\`		Z\_1 = \(∑\+i 
              Exp[\(-β\)\ E\_i\%\((1)\)]\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Z\_2 = ∑\+j 
            Exp[\(-β\)\ E\_j\%\((2)\)]\)\)
\(TextForm\`		F\_1 = \(\(-k\_B\) 
          T\ Log[Z\_1]\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ F\_2 = \(-k\_B\) T\ Log[Z\_2]\)\)

Hence, F, U, and S are extensive variables for weakly interacting systems.  This argument can be extended to a system of N identical noninteracting constituents with the important result

\(TextForm\`noninteracting\ \ ⟹\ \ \ Z\_N = \(\((Z\_1)\)\^N\ \ \ ⟹\ \ \ F\  = \ \(-N\)\ \(k\_B\) 
          T\ Log[Z\_1]\)\)

where (Subscript[Z, 1]) is the single-particle partition function.

Example: binary system

Suppose that each member of a system of N noninteracting constituents can occupy two states with energies ε and ε + Δ, where ε is the ground-state energy and Δ is the energy gap.  The single-particle partition function is then

\(TextForm\`Z\_1 = \(Exp[\(-\(ε\/\(\(k\_B\) T\)\)\)] + 
          Exp[\(-\(\(ε + Δ\)\/\(\(k\_B\) 
                    T\)\)\)] = \(e\^\(\(-β\)\ ε\)\) \((1 + e\^\(-y\))\)\)\)

where y = β Δ = Δ/
              (Subscript[k, B])  T is the reduced excitation energy defined as the ratio between excitation and thermal energies.  The principal thermodynamic functions are

F = 
      N (ε -  (Δ/y) Log[
                1 + (e^(-y))])
\(TextForm\`U = 
      N \((ε\  + \ Δ\/\(1 + e\^y\))\)\)
S = 
      N (Subscript[k, B]) ((y/(1 + (e^y))) + 
            Log[1 + (e^(-y))])

where the energy offset N ε that appears in both F and U has no thermodynamical significance because it is not coupled to temperature.  Thus, the heat capacity

\(TextForm\`C\_V = 
      N\ \(k\_B\) \(\(y\^2\) e\^y\)\/\((1 + e\^y)\)\^2\)

depends only upon the reduced excitation energy and is independent of the ground-state energy.  With a little algebra one can demonstrate that these results are consistent with those obtained earlier using the microcanonical ensemble, but you will probably agree that the derivation is easier using the canonical ensemble.  The microcanonical ensemble is useful for deriving general relationships, such as the form of the canonical probability distribution, but applications are usually easier to make with the canonical ensemble.

Example: ideal paramagnetism

In this example we compare and contrast the quantum mechanical and classical theories of ideal paramagnetism.  The orientational energy for a permanent magnetic dipole moment (Overscript[μ,  ⇀])  in an external magnetic field (Overscript[B,  ⇀])  is \(TraditionalForm\`ε = \(-\(μ\& ⇀ \)\)·\(B\& ⇀ \)\).  For atomic physics the temperature scale for paramagnetic behavior is determined by the Bohr magneton

(Subscript[μ, 0]) -> (Subscript[μ, B]) = ((e ℏ)/(2 (Subscript[m, e]) 
                c))  ⟹  ((Subscript[μ, B])/(Subscript[k, B])) =  
        0.672 Kelvin/Tesla

while for nuclear physics the nuclear magneton and corresponding temperature scale is smaller by the ratio of electron to proton masses:

(Subscript[μ, 0]) -> (Subscript[μ, N]) = ((e ℏ)/(2 (Subscript[m, p]) 
                c))  ⟹  ((Subscript[μ, N])/(Subscript[k, B])) = 
        0.000366 Kelvin/Tesla

In the quantum mechanical theory, the magnetic moment can be expressed as (Overscript[μ,  ⇀])  = 
        g (Subscript[μ, 0]) (Overscript[j,  ⇀])  where (Subscript[μ, 0]) is the unit of magnetic moment (Bohr or nuclear magneton, as appropriate), (Overscript[j,  ⇀])  is the total angular momentum, usually containing both spin and orbital contributions, and g is a gyromagnetic ratio describing the internal structure of the particle or atom.  Recognizing that the component of angular momentum along the magnetic field is quantized, the discrete spectrum of orientational energy levels becomes (Subscript[ε, m]) = -(m/j) μ B where μ is the magnetic moment in the aligned state with m = (Subscript[j, z]) = j.  Thus, it is useful to define y =  μ B/(Subscript[k, B])  T as the ratio between orientational and thermal energies such that the single-particle partition function becomes

(Subscript[Z, 1]) = (∑_(m = -j)^j) Exp[m y/j] = 
        Sinh[((2  j + 1)/(2  j)) y]/Sinh[(y/(2  j))]

The principal thermodynamic functions can now be evaluated straightforwardly.  Some of these functions are summarized below and the gory details can be found in paramag.nb.

F = -N (Subscript[k, B]) 
        T Log[Sinh[((2  j + 1)/(2  j)) y]/Sinh[(y/(2  j))]]
\(TextForm\`M = \(\(-\((∂F\/∂B)\)\_T\) = 
        N\ μ\ \((\((2\ j + 1)\)\/\(2\ j\)\ Coth[\(\((2\ j + 1)\)\ y\)\/\(2\ j\)]\  - \(1\/\(2\ j\)\) 
                Coth[y\/\(2\ j\)]\ )\)\)\)
U = -M B
\(TextForm\`C\_B = \(\((∂U\/∂T)\)\_B = 
        N\ k\_B\ \ \((\ \((\(y\/\(2  j\)\) Csch[y\/\(2\ j\)]\ )\)\^2\  - \ \((\(\((2\ j + 1)\) y\)\/\(2  j\)\ Csch[\(\((2\ j + 1)\)\ y\)\/\(2\ j\)]\ )\)\^2\ )\)\)\)

One can easily demonstrate that our previous results for spin (1/2) are recovered when j -> (1/2).

It is instructive to compare these results with a classical theory for which the orientation of the magnetic dipole is a continuous function of angle, such that ε = -μ B cosθ.  The summation over states becomes an integral over the unit sphere, such that

\(TextForm\`Z\_1 = \(2  π \(∫\_\(-1\)\%1dx\ e\^\(x\ y\)\) = 4  π\ Sinh[y]\/y\)\)

where the alignment variable x = cosθ is uniformly distributed between {-1, 1}.  The corresponding thermodynamic functions constructed in like manner are:

F = -N (Subscript[k, B]) 
        T Log[4  π Sinh[y]/y]
M = -(Subscript[((∂(F/∂)B)), T]) = 
        N μ ( Coth[y]  -  (1/y) )
U = -M B
(Subscript[C, B]) = (Subscript[((∂(U/∂)T)), B]) = 
        N (Subscript[k, B]) (1 - ((((y/Sinh)[y]))^2))

Notice that for fixed μ the quantum mechanical and classical formulas for M and (Subscript[C, B]) are identical in the limit j -> ∞.  We must hold μ fixed because in the classical theory there μ is independent of j.  For the quantum mechanical theory one can imagine that this limit is taken with {g -> 0, j -> ∞} such that μ = g (Subscript[μ, 0]) j remains constant.

The figure below compares the classical magnetization with quantum mechanical results for small and large spins.  For all cases saturation is observed for large y, corresponding to large B or small T, but the quantum mechanical theory shows that saturation is approached more quickly for small j than predicted by the classical theory.  For both theories one finds that the magnetic susceptibility for large temperatures or small fields is described by Curie's law

\(TextForm\`χ = \(-\((∂M\/∂B)\)\_T\) ≈ χ\_0\/T\)

but the classical and quantum mechanical Curie constants

\(TextForm\`classical\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ⟹\ \ χ\_0 = \(N\ μ\^2\)\/\(3  k\_B\)\)
\(TextForm\`quantum\ mechanical\ \ ⟹\ \ χ\_0 = \(\(N\ g\ \(μ\_0\^2\) j \((j + 1)\)\)\/\(3  k\_B\) = \(\(N\ μ\^2\)\/\(3  k\_B\)\) \(j + 1\)\/j\)\)

differ by a factor of (j + 1)/j, which is 3 for spin (1/2).  The spin-dependent factor in (Subscript[χ, 0]) is simply the quantum mechanical expectation of ⟨(j^2)⟩ = 
        j(j + 1).  Although these models of the magnetization are qualitatively consistent, the quantization of orientation of the magnetic dipole moment strongly affects the Curie constant for small spins.

The figure below compares the magnetic heat capacities for the classical and quantum mechanical theories.  Classically one finds that the heat capacity saturates for large y, but quantum mechanically one finds that the heat capacity approaches zero for large y.  The difference between these theories is especially great for small spins.  As the spin increases the quantum mechanical heat capacity stays near the classical limit longer, but eventually it peaks and decreases, failing to maintain saturation.  Thus, the Schottky heat-capacity anomaly we discussed for binary systems is present for higher spins also, differing in shape and scale but not in character.  Why does (Subscript[C, B]) differ from the classical limit more strongly than M?  A classical system can absorb infinitesimally small amounts of heat because its energy spectrum is continuous, but because the quantum mechanical energy spectrum is quantized energy can be absorbed only in multiples of the energy spacing Δε = 
        g (Subscript[μ, 0]) B.  Therefore, the gap between the lowest two energy levels inhibits heat transfer when (Subscript[k, B]) T is appreciably smaller than Δε.  When practically all atoms are found in the lowest energy level, the gap to the next level is more important than the spectrum of higher levels.  If there is a finite gap, the thermodynamics for very low temperatures resembles that of the two-state system.

Example: lattice vibrations

A crystal consists of N atoms arranged upon a lattice.  The atoms may be considered distinguishable by their lattice positions.  Let (Subscript[x, i])[t] denote one of the 3  N coordinates specifying the atomic positions and let

\(TextForm\`H\  = \ \(1\/2\) 
          m\ \(∑\+\(i = 1\)\%\(3  N\)\(x\& . \)\_i\^2\)\  + \ 
        V[x\_1, ⋯, x\_\(3  N\)]\)

represent the total kinetic and potential energies of the crystal.  Assuming that atomic vibrations are limited to small displacements from the equilibrium positions \(TraditionalForm\`x\&_\_i\), the potential energy can be expanded to second order according to

\(TextForm\`V[x\_1, ⋯, x\_\(3  N\)]\  ≈ \ 
      V\&_[x\&_\_1, ⋯, 
          x\&_\_\(3  N\)]\  + \ \(1\/2\) \(∑\+\(i, j = 1\)\%\(3  N\)\(\((∂\^2 V\/\(∂x\_i ∂x\_j\))\)\_\(x\&_\_1, ⋯, x\&_\_\(3  N\)\)\) \((x\_i - x\&_\_i)\) \((x\_j - 
                  x\&_\_j)\)\)\ \  + \ ⋯\)

We assume that the constant term \(TraditionalForm\`V\&_ = \(-N\)\ ϕ\) is proportional to the number of atoms, where ϕ is the average binding energy per atom, and define a curvature matrix

\(TextForm\`\(\(α\_\(i, 
          j\)\)\(\ \)\(=\)\(\ \)\(\((∂\^2 V\/\(∂x\_i ∂x\_j\))\)\_\(x\&_\_1, ⋯, 
          x\&_\_\(3  N\)\)\)\(\ \ \ \ \ \ \ \ \ \)\)\)

such that

\(TextForm\`V[x\_1, ⋯, 
        x\_\(3  N\)]\  ≈ \(-N\)\ ϕ\  + \ \(1\/2\) \(∑\+\(i, j = 1\)\%\(3  N\)α\_\(i, j\)\ ξ\_i\ ξ\_j\)\ \ \  + \ ⋯\ \ ⟹\ \ \ H\  ≈ \(-N\)\ ϕ\  + \ \(1\/2\) 
          m\ \(∑\+\(i = 1\)\%\(3  N\)\(ξ\& . \)\_i\^2\)\  + \ \(1\/2\) \(∑\+\(i, j = 1\)\%\(3  N\)α\_\(i, j\)\ ξ\_i\ ξ\_j\)\)

where displacements from equilibrium are denoted \(TraditionalForm\`ξ\_i\  = \ x\_i - x\&_\_i\).  It is useful to treat (Subscript[ξ, i]) as a component of a vector ξ, such that the model Hamiltonian can be expressed in the form

H = -N ϕ  + (1/2)( m (OverDot[ξ]),·(OverDot[ξ]), + ξ·α·ξ )

The matrix α is clearly symmetric and must also be positive-definite because the potential energy was expanded around the stable equilibrium configuration.  Therefore, the potential energy can be diagonalized by an orthogonal transformation S, such that

(OverTilde[S]) α S = λ

where λ is a diagonal matrix with positive eigenvalues \(TraditionalForm\`λ\_\(i, j\) = \(λ\_i\) δ\_\(i, j\)\) and where (OverTilde[S]) is the transpose of S such that \(\(S\&~\)\ S\)=1.  Defining normal coordinates

ξ = S·q   ⟹   ξ·α·ξ =  q· λ·q = (∑_(i = 1)^(3  N))(Subscript[λ, i]) 
                ((Subscript[q, i])^2)

and momenta

p = m (OverDot[q])  ⟹    m (OverDot[ξ]),·(OverDot[ξ]) = ((p·p)/(2  m))

the Hamiltonian reduces to

\(TextForm\`H\  = \ \(-N\)\ ϕ\  + \ ∑\+\(i = 1\)\%\(3  N\)\((p\_i\^2\/\(2  m\) + \ \(1\/2\) m\ \(ω\_i\^2\) 
                q\_i\^2)\)\)

where the frequencies are identified by (Subscript[λ, i]) = m ((Subscript[ω, i])^2).  Thus, expressing the displacements in terms of normal modes of vibration reduces the Hamiltonian to a sum of independent terms, each of which has the form of a harmonic oscillator.  There are 3  N normal modes with a spectrum of vibrational frequencies {(Subscript[ω, i]), i = 1, 3  N} that depends in a complicated manner upon the interatomic potential that binds the crystal together.  Although it may be difficult or impossible to compute these frequencies from first principles, the excitation spectrum is readily measured by neutron scattering or other means.

The analysis has been essentially classical to this point, but quantization of the Hamiltonian for a harmonic oscillator should be quite familiar.  We simply replace the classical Hamiltonian function by the quantum mechanical operator

\(TextForm\`H\&^\  = \ \(-N\)\ ϕ\  + \ ∑\+\(α = 1\)\%\(3  N\)\(ℏω\_α\) \((n\&^\_α + 
                1\/2)\)\)

where \(TraditionalForm\`n\&^ = \(a\&^\_α\^†\) 
          a\&^\_α\) is the number operator and where \(TraditionalForm\`a\&^\_α\^†\) and  \(TraditionalForm\`a\&^\_α\) are creation and destruction operators for quantized excitations of mode α.  Vibrational quanta are known as phonons, named by analogy with electromagnetic quanta known as photons.  The thermodynamic behavior of a wide variety of physical systems can be analyzed in terms of quantized excitations of various types.  If one can approximate the Hamiltonian by a sum of independent normal modes, the subsequent thermodynamic analysis is typically very similar to that for lattice vibrations.  Hence, this example represents a very important prototype.

The canonical partition function for this separable Hamiltonian of this type

\(TextForm\`Z\  = \ \(Tr[\ 
          Exp[\(-β\)\ H\&^]\ ]\  = \ e\^\(β\ N\ ϕ\)\ \ \(∏\+\(α = 1\)\%\(3  N\)Z\_α\)\)\)

factorizes into single-mode partition functions

\(TextForm\`Z\_α\  = \(∑\+\(n\_α = 0\)\%∞ 
            Exp[\(-β\)\ \(ℏω\_α\) \((n\_α + 
                  1\/2)\)]\  = \ 
        Exp[βℏω\_α/2]\/\(Exp[βℏω\_α] - 1\)\)\)

where each mode can carry an arbitrarily large number of phonons.  The internal energy

\(TextForm\`U = \(\(-\(\(∂ln\ Z\)\/∂β\)\) = \ \(-N\)\ ϕ\  + \ ∑\+\(α = 1\)\%\(3  N\)\(ℏω\_α\) \((n\&_\_α + 1\/2)\)\)\)

separates into a temperature-independent zero-point and binding energy plus the sum of the excitation energies for each mode carrying an average number of phonons given by

\(TextForm\`n\&_\_α = 
      1\/\(Exp[βℏω\_α] - 1\)\)

To complete this model, we must determine the spectrum of normal modes, {(Subscript[ω, α]), α = 1, 3  N}.  If we assume that the spectrum is essentially continuous, we can replace the discrete summation by an integral of the form

\(TextForm\`U = \ \(-N\)\ ϕ\  + \ ∫dω\ g[ω]\ ℏω\/\(Exp[βℏω] - 1\)\)

where g[ω] is the density of states such that g[ω] dω is the number of states between ω and ω + dω.  The density of states must be normalized according to

∫dω g[ω]  =  
      3  N

such that the total number of states represents three degrees of freedom per atom.

The simplest version of this model, first proposed by Einstein, assumes that all frequencies can be approximated by a single average value (Subscript[ω, E]).  It is then convenient to defined a reduced temperature τ = T/(Subscript[T, E]) where the Einstein temperature is given by ℏ(Subscript[ω, E]) = (Subscript[k, B]) (Subscript[T, E]).  Evaluating the internal energy

\(TextForm\`g[ω] = \(3  
          N\ δ[ω - ω\_E]\ \ \ ⟹\ \ \ U = 3  N\ \(ℏω\_E\) \((1\/\(Exp[T\_E\/T] - 1\) + 1\/2)\)\  - \ 
          N\ ϕ\)\)

we find that the heat capacity takes the form

(Subscript[C, V]) = (3/4) N (Subscript[k, B]) (((((Subscript[T, E])/T)))^2) 
        Csch[((Subscript[T, E])/(2  T))]^2

With a little algebra one can show that

\(TextForm\`T -> \(∞\ \ ⟹\ \ C\_V -> 3  N\ k\_B\)\)
T -> 
      0    ⟹ (Subscript[C, V]) ≈ 
        3  N (Subscript[k, B]) (((((Subscript[T, E])/T)))^2) 
          Exp[-((Subscript[T, E])/T)]

Thus, at high temperature one obtains the classical equipartition theorem but at low temperature the heat capacity is exponentially suppressed.

Debye proposed a more realistic model of the energy spectrum in which g[ω] ∝ (ω^2) for ω ≤ (Subscript[ω, D]).  The constant of proportionality is determined easily using the normalization condition

\(TextForm\`a \(∫\_0\%\(ω\_D\)\(ω\^2\) dω\) = \(3 
           N\ \ ⟹\ \ a = \(\(\(9  
                  N\)\/ω\_D\^3\)\ ⟹\ \ g[ω] = 
          9  N\ \(ω\^2\/ω\_D\^3\) Θ[ω\_D - ω]\)\)\)

where the Heaviside truth function Θ[x] is unity for x ≥ 0 and zero otherwise.  For this model the internal energy can be expressed in the form

\(TextForm\`U\  + \ 
        N\ ϕ = \(\(\(9  
                N\ ℏ\)\/ω\_D\^3\) \(∫\_0\%\(ω\_D\)dω\ ω\^3\/\(Exp[βℏω] - 1\)\)\  = \ 
        9  N\ ℏ\ \(ω\_D\) \(\((T\/T\_D)\)\^4\) \(∫\_0\%\(T\_D/T\)dy\ y\^3\/\(e\^y - 1\)\)\)\)

where the Debye temperature, (Subscript[T, D]), is related to the Debye cut-off frequency, (Subscript[ω, D]), by (Subscript[k, B]) (Subscript[T, D]) = ℏ(Subscript[ω, D]).  Although this integral can be evaluated in terms of named functions, we refer the reader to the notebook debye.nb because those functions will not be familiar to most readers.   For the present purposes it is sufficient to note that the low-temperature behavior can be obtained by extending the range of integration to ∞ and using the definite integral

\(TextForm\`∫\_0\%∞dy\ y\^3\/\(e\^y - 1\) = π\^4\/15\)

to obtain

\(TextForm\`T ≪ 
      T\_D\ \ ⟹\ \ U + 
        N\ ϕ\  ≈ \ \(\(\(\ \)\(3  π\^4\)\)\/5\) 
        N\ ℏ\ \(ω\_D\) \((T\/T\_D)\)\^4\ \ ⟹\ \ \ C\_V ≈ \(\(12  π\^4\)\/5\) 
        N\ \(k\_B\) \((T\/T\_D)\)\^3\)

The Debye model predicts the low-temperature heat capacity is governed by a power law rather than by an exponential function becauses its continuous energy spectrum does not feature an energy gap.

The low frequency or long wavelength excitations represent collective modes in which the displacements of neighboring atoms are similar — these are acoustical waves with correlated motions.  Excitation quanta are smaller for low frequency than for high frequency, but the number of collective modes decreases as the correlation between atomic motions increases.  Therefore, although the excitation spectrum is continuous and the crystal can accept arbitrarily small excitation energy (in the limit N -> ∞), the heat capacity is suppressed at low temperature because the fraction of the available modes that are thermodynamically active is proportional to (((T/(Subscript[T, D])))^3).  We obtain a power-law instead of exponential suppression because there is no gap in the excitation spectrum for collective modes of oscillation.  The figure below compares some of the earliest low-temperature data for the capacity in an insulating crystal with a fit based upon the predicted (((T/(Subscript[T, D])))^3) behavior.  The data are fit very well by this power law and the fitted value of (Subscript[T, D]) is consistent with calculations based upon either elastic moduli or upon the acoustical excitation spectrum.  Conversely, high-frequency modes represent essentially independent oscillations of individual atoms about their equilibrium positions with a limiting frequency that is inversely proportional to the lattice spacing.  Therefore, at high temperature the thermodynamics of the system are well approximated by a system of independent oscillators.

Ref.: P. H. Keesom and N. Pearlman, Phys. Rev. 91, 1354 (1953)

It is also of interest to compare these results with those of the corresponding classical theory.  A classical oscillator can accept infinitesimal excitation energy, such that the single-mode partition function is represented by an integral of the form

\(TextForm\`Z\_α\  = \ \(∫\_0\%∞\( e\^\(\(-β\)\ ε\)\) dε\  = \ \(k\_B\) T\)\)

instead of a discrete sum over quanta.  Thus, the average energy per mode

\(TextForm\`ε\&_\_α\  = \ \(\(-\(∂Log[
                  Z\_α]\/∂β\)\)\  = \ \(k\_B\) 
          T\)\)

is independent of frequency and the internal energy

\(TextForm\`U\  + 
        N\ ϕ\  = \ \(∫\_0\%∞dω\ g[ω] ε\&_[ω]\  = \ \(3  N\ \(k\_B\) 
            T\ \ \ ⟹\ \ \ C\_V = 
          3  N\ k\_B\)\)\)

is simply the total number of modes times the average energy per mode.  Therefore, the classical heat capacity for lattice vibrations is independent of temperature.  This result, known as the Dulong-Petit law, is an example of the classical equipartition theorem.  However, the low-temperature data for insulating solids follow the (T^3) behavior of the Debye theory and disagree with the classical prediction.  The essential difference between the two theories is the discretization of the single-mode energy spectrum in the quantum theory.

Unfortunately, there are several serious inconsistencies in the classical theory.  First, because the partition function should be dimensionless we should have included a density of states factor in our integral.  In this case, the density of states would simply be constant with units of inverse energy, but classical physics offers no natural energy scale by which to determine its value.  Consequently, the classical entropy is undetermined because it depends upon this scale parameter.  More importantly, the number of modes for a classical system is infinite.  Although we assumed that the number of modes is limited to the 3  N vibrational modes of N atoms, a classical mass can always be decomposed into smaller constituents bound together by internal forces.  If an object appears to be rigid, that just means that its internal binding forces are strong.  However, no matter how strong those binding forces are, it should always be possible to induce small-amplitude oscillations against those forces.  To be more specific, we know that atoms consist of electrons bound to a central nucleus by electromagnetic forces.  Classically we would expect small-amplitude vibrations about the equilibrium configuration of those constituents.  Furthermore, the nucleus itself consists of protons and neutrons which should also exhibit vibrational modes.  The nucleons themselves are composite objects which should possess a spectrum of normal modes.  Therefore, the number of normal modes is much larger than 3  N and, if one takes classical physics seriously, should probably be infinite!  In order to obtain finite results for heat capacities, classical theories must limit themselves to a finite set of active modes even though the theory offers no dynamical method for suppressing stiff internal modes.  There is no natural energy scale in the theory and all modes are capable of accepting energy even when their vibrational amplitudes are infinitesimal.  Quantum mechanics, on the other hand, offers a natural method to decide whether a mode is active or passive.  Due to the quantization of energies, oscillators whose quanta are larger than the typical thermal energy, (Subscript[k, B]) T, are inactive.  Thus, all modes with ℏ ω ≫ (Subscript[k, B]) T are frozen out and may be safely ignored.  Consequently, the calculation of heat capacity need only include those modes that are potentially active.

Quantization is essential to the success of statistical mechanics.  Although many textbooks begin with classical statistical mechanics, that approach is fundamentally flawed.  Classical physics cannot compute entropy or heat capacity without appealing to quantum mechanics, or ad hoc prescriptions, to count the number of states and to limit the number of active modes.  Therefore, we have chosen to develop statistical mechanics from a quantum mechanical perspective from the outset.  Nevertheless, we often find that semiclassical models can be useful and provide valuable insight when derived as a suitable limit from quantum mechanics.  We use quantum mechanics to evaluate the density of states and to decide which modes are active.  We then use classical methods to simplify the calculations, when appropriate.  This approach will be developed in semiclassical.nb.

Relationship between canonical and microcanonical ensembles

In the canonical ensemble, the probability that a system will be found with energy E between E and E + dE is

\(TextForm\`P[
          E] dE = \(\(Z\^\(-1\)\) \(e\^\(\(-β\)\ E\)\) 
          g[E] dE\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Z = ∫dE\ g[
              E]\ e\^\(\(-β\)\ E\)\)\)

where g[E] is the density of states.  Normally g[E] is a rapidly increasing function of energy, resembling (E^N), whereas the Boltzmann factor (e^(-β E)) is a rapidly decreasing function.  Thus, the product g[E] (e^(-β E)) for a large system is usually sharply peaked about the most probable energy \(TraditionalForm\`E\&~\).   The most probable energy is determined simply by setting the derivative wrt energy to zero

\(TextForm\`\((\(∂ln\ P[E]\)\/∂E)\)\_\(E = E\&~\) = \(0\ \ \ \ ⟹\ \ \ \ β = \((\(∂ln\ g[E]\)\/∂E)\)\_\(E = E\&~\)\)\)

with both Z and β treated as constants.  The parameter β = ((((Subscript[k, B]) (Subscript[T, R])))^(-1)) is fixed by the temperature of the reservoir, (Subscript[T, R]), which is unaffected by energy exchange with the system of interest, whose heat capacity is much smaller.  If we interpret \(TraditionalForm\`1\/T = \(k\_B\) \(∂ln\ g\)\/∂E\) as the inverse temperature of the system, we find that maximizing the probability with respect to the internal energy of the system is equivalent to the equilibrium condition T = (Subscript[T, R]).  Therefore, the fixed energy in the microcanonical ensemble is equivalent to the most probable energy in the canonical ensemble.  Similarly, the width of the peak in the canonical energy distribution is equivalent to the interval used by the microcanonical ensemble to bound the energy.

We can also compare the mean energy U with the most probable energy \(TraditionalForm\`E\&~\).

\(TextForm\`U = \(∫dE\ P[
              E]\ E\ \  = \ \(\(Z\^\(-1\)\) \(∫dE\ g[
                  E] e\^\(\(-β\)\ E\)\)\  = \(-\(∂\/∂β\)\) ln\ Z\)\)\)

Near \(TraditionalForm\`\(E\&~\)\), we expand

\(TextForm\`Log[\ 
        g[E] e\^\(\(-β\)\ E\)\ ] ≈ \ 
      Log[\ g[E\&~] e\^\(\(-β\)\ E\&~\)\ ]\  + \ \(1\/2\) \(\((E - E\&~)\)\^2\) \((\(∂\^2\/∂E\^2\) Log[\ g[E] e\^\(\(-β\)\ E\)\ ]\ )\)\_\(E = E\&~\)\  + \ ⋯\)

and define

\(TextForm\`σ\^\(-2\) = \(\(-\)\(\((\(∂\^2\/∂E\^2\) Log[\ g[E] e\^\(\(-β\)\ E\)\ ])\)\_\(E = 
              E\&~\)\)\(\ \)\)\)

so that

\(TextForm\`P[E] ≈ 
      P[E\&~]\ Exp[\(-\(\((E - E\&~)\)\^2\/\(2  σ\^2\)\)\)]\)

displays a Gaussian shape in the vicinity of the peak.  Expressing the density of states in terms of entropy, the variance in energy becomes

\(TextForm\`\(∂\^2\/∂E\^2\) 
        Log[\ g[E] e\^\(\(-β\)\ E\)\ ]\  = \ \(\(1\/k\_B\) \((∂\^2 S\/∂E\^2)\)\ \ \ \ ⟹\ \ \ σ\^\(-2\) = \(\(-\(1\/k\_B\)\) \(\((∂\^2 S\/∂E\^2)\)\_\(E = 
                E\&~\)\)\(\ \)\)\)\)

If the width σ is sufficiently narrow that higher-order terms may be neglected, then \(TraditionalForm\`E\&~ ≈ U\).  Furthermore, the width of the energy distribution is related to thermodynamic properties by

\(TextForm\`σ\^2 = \(\(-k\_B\) \((∂\^2 S\/∂E\^2)\)\_\(E = E\&~\)\%\(-1\)\  = \ \(\(-k\_B\) \((\(∂\/∂U\) 1\/T)\)\^\(-1\) = \ \(k\_B\) \(T\^2\) 
            C\_V\)\)\)

where (Subscript[C, V]) = ∂(U/∂)T is the isochoric heat capacity.  Since (Subscript[C, V]) ∝ N and U ∝ N, normally, we expect that σ ∝ 
        (N^(1/2)) ⟹(σ/U) ∝ 
        (N^(-1/2)) so that for large systems the energy distribution becomes very sharp, practically a δ-function, except near a phase transition or other divergence in (Subscript[C, V]).

Finally, we note that

\(TextForm\`g[
          U] e\^\(\(-β\)\ U\) = \(Exp[
            ln\ g] e\^\(\(-β\)\ U\) = \(Exp[\(-β\) \((U - T\ S)\)]\  = \ e\^\(\(-β\)\ F\)\ \ ⟹\ \ \ P[
                E\&~] ≈ \ e\^\(\(-β\)\ F\)\)\)\)

where F = U - T S is the Helmholtz free energy.  Therefore, the canonical probability distribution becomes

\(TextForm\`P[E] ≈ \(e\^\(\(-β\)\ F\)\) 
          Exp[\(-\(\((E - U)\)\^2\/\(2  σ\^2\)\)\)]\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ σ\^2 = \(k\_B\) \(T\^2\) 
        C\_V\)

for E near \(TraditionalForm\`\(E\&~\) = U\).  Furthermore, the partition function becomes

\(TextForm\`Z = \(∫dE\ g[
                E]\ e\^\(\(-β\)\ E\)\  ≈ \ ∫dE\ \(e\^\(\(-β\)\ F\)\) 
              Exp[\(-\(\((E - U)\)\^2\/\(2  σ\^2\)\)\)]\ \  = \ \(e\^\(\(-β\)\ F\)\) \(\@\(2  π\)\) σ\)\)

so that

ln Z  ≈  -β F  +  (1/2) 
          Log[2  π (Subscript[k, B]) (T^2) (Subscript[C, V])]

where the second term originates from the normalization of the probability distribution.  However, since F ∝ N the second term, which is proportional to ln N, may be neglected for large N.  Hence, we conclude that \(TraditionalForm\`Z ≈ e\^\(\(-β\)\ F\)\) and that

P[E] ≈ (Z^(-1)) 
        Exp[-((((E - U))^2)/(2  (σ^2)))]

near the peak.

Therefore, the canonical and microcanonical ensembles provide essentially equivalent statistical description of the thermodynamics of macroscopic systems.  The two probability distributions compared below are equivalent for all practical purposes.  For large systems the widths of these distributions become extremely small compared with U, so that the difference between Gaussian and square distributions becomes negligible.

Grand canonical ensemble

Derivation

Consider a small system in thermal and diffusive contact with a much larger reservoir with which it may exchange both heat and particles.  The combined system is isolated, so that the total energy E and total particle number N is conserved.  These are shared according to

\(TextForm\`	
    E\_R + E\_i = 
      E\_T\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ E\_i ≪ E\_R\)
\(TextForm\`	
    N\_R + N\_i = 
      N\_T\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ N\_i ≪ 
        N\_r\)

The probability that the sample is found in a particular state i is proportional to the total number of states available to the reservoir, such that

(Subscript[P, i]) = 
      C (Subscript[Ω, R])[(Subscript[E, T]) - (Subscript[E, i]), (Subscript[N, T]) - (Subscript[N, i])]

Expanding ln (Subscript[Ω, R]) for small ((Subscript[E, i]), (Subscript[N, i])), we find

ln (Subscript[Ω, R])[(Subscript[E, T]) - (Subscript[E, i]), 
          (Subscript[N, T]) - (Subscript[N, i])] ≈  
      ln (Subscript[Ω, R])[(Subscript[E, T]), 
            (Subscript[N, T])]  -  β ((Subscript[E, i]) - μ (Subscript[N, i]))

where the expansion coefficients

\(TextForm\`β = \(\((\(∂ln\ Ω\_R\)\/∂E\_R)\)\_\(E\_R = E\_T\)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ β\ μ = \(-\((\(∂ln\ Ω\_R\)\/∂N\_R)\)\_\(N\_R = N\_T\)\)\)\)

are defined by recognizing that the fundamental relation for S = (Subscript[k, B]) ln (Subscript[Ω, R]) takes the form

\(TextForm\`T dS = \(dU + 
          p dV - μ dN\ \ ⟹\ \ \ \ \ \((∂S\/∂U)\)\_\(V, N\) = \(1\/T\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \((∂S\/∂N)\)\_\(U, V\) = \(-\(μ\/T\)\)\)\)\)

It is not necessary to include subscripts on T or μ because thermal equilibrium requires those intensive parameters to be the same for both the sample and the reservoir.  Hence, the reservoir multiplicity function takes the form

(Subscript[Ω, R])[(Subscript[E, T]) - (Subscript[E, i]), 
        (Subscript[N, T]) - (Subscript[N, i])] ≈ (Subscript[Ω, R])[(Subscript[E, T]), 
          (Subscript[N, T])] Exp[-β ((Subscript[E, i]) - μ (Subscript[N, i]))]

Therefore, the population of states of a system in diffusive contact with a reservoir of energy and particles is governed by the grand canonical distribution

\(TextForm\`P\_i = \(\(𝒵\^\(-1\)\) 
          Exp[\(-β\) \((E\_i - μ\ N\_i)\)]\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ 𝒵 = \(\(∑\+i\)\(Exp[\(-β\) \((E\_i - μ\ N\_i)\)]\)\(\ \)\)\)\)

where 𝒵 is known as the grand partition function.  The mean number of particles in the system is determined by the ensemble average

\(TextForm\`N = \(\(𝒵\^\(-1\)\) \(∑\+i 
                Exp[\(-β\) \((E\_i - μ\ N\_i)\)]\ N\_i\)\  = \ \(k\_B\) T \((\(∂ln\ 𝒵\)\/∂μ)\)\_\(T, V\)\)\)

The bridge to thermodynamics is most conveniently developed through the grand potential

𝒢[T, V, μ]  =  F - μ N  =  
        U  - T S  -  μ N

The grand potential can be simplified using the Euler relation

\(TextForm\`U = \(T\ S\  - \ 
          p\ V\  - \ μ\ N\ \ \ ⟹\ \ 𝒢 = \(-p\)\ V\)\)

In terms of its natural independent variables, the fundamental relation for 𝒢 now takes the form

\(TextForm\`d𝒢 = \(-S\) dT - p dV - N dμ\)

from which we identify

\(TextForm\`S = \(\(-\((∂𝒢\/∂T)\)\_\(V, μ\)\)\ \ \ \ \ \ \ \ \ \ p = \(\(-\((∂𝒢\/∂V)\)\_\(T, μ\)\)\ \ \ \ \ \ \ \ \ N = \(-\((∂𝒢\/∂μ)\)\_\(T, V\)\)\)\)\)

The relationship for N permits the identifications

\(TextForm\`𝒢 = \(\(-k\_B\) 
          T\ ln\ 𝒵\ \ \ \ ⟹\ \ \ \ \(\(p\ V\)\/\(\(k\_B\) T\)\) = ln\ 𝒵\)\)

The validity of this identification depends upon recovering the thermodynamic expressions for S and p from the statistical expressions for the grand partition function.  Differentiating 𝒢 with respect to T, we find

\(TextForm\`∂𝒢\/∂T = \(𝒢\/T - \(k\_B\) 
            T\ \(∂ln\ 𝒵\)\/∂T = \(1\/T\) \((𝒢 + \(∂ln\ 𝒵\)\/∂β)\)\)\)

From the definition of 𝒵, we find

\(TextForm\`\(∂ln\ 𝒵\)\/∂β = \(\(-\((U - μ\ N)\)\ \ \ ⟹\ \ \(∂𝒢\/∂T\)\) = \(\(1\/T\) \((𝒢 - 
                U + μ\ N)\) = \(-S\)\)\)\)

in agreement with the thermodynamic relationship.  Similarly, differentiating 𝒢 with respect to V, we find

∂(𝒢/∂)V = -(Subscript[k, B]) 
        T ((∂ln 𝒵)/∂)V

However, ln 𝒵 depends upon V only through its dependence on the energy levels (Subscript[E, i]), whereby

\(TextForm\`\((\(∂ln\ 𝒵\)\/∂V)\)\_\(T, μ\) = \ \(\(-\(β\/𝒵\)\) \(∑\+i 
                Exp[\(-β\) \((E\_i - μ\ N\_i)\)]\ ∂E\_i\/∂V\)\  = \ \(β ⟨\(-\(∂E\/∂V\)\)⟩\  = \ β\ p\)\)\)

Thus, the thermodynamic relationship for pressure,

\(TextForm\`p = \(\(k\_B\) 
          T \((\(∂ln\ 𝒵\)\/∂V)\)\_\(T, μ\) = \(-\((∂𝒢\/∂V)\)\_\(T, μ\)\)\)\)

is also recovered from the grand partition function.  Finally, we note that

\(TextForm\`U - μ\ N = \(\(\(∂ln\ 𝒵\)\/∂β\)\ \ ⟹\ \ U = \(μ\ N + \(∂\/∂β\) \((β\ 𝒢)\) = μ\ N + 𝒢 + T\ S\)\)\)

reproduces the Euler relation.  Therefore, the standard thermodynamic relationships can be extracted from the grand partition function through the identification of the grand potential as

\(TextForm\`𝒢[T, 
        V, μ]\  = \ \(U\  - 
          T\ S\  - \ μ\ N\  = \ \(\(-p\)\ V\  = \ \(\(-k\_B\) 
              T\ ln\ 𝒵\ \ ⟹\ \ \(\(p\ V\)\/\(\(k\_B\) T\)\) = ln\ 𝒵\)\)\)\)

Alternatively, the grand canonical probability distribution can be derived by maximizing the disorder within an ensemble of isolated systems with specified mean energy and particle number.

\(TextForm\`dS = \(0\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ⟹\ \ \(∑\+i\((1 + 
                  ln\ P\_i)\) dP\_i\) = 
        0\)\)
U = (∑_i) (Subscript[P, i]) 
            (Subscript[E, i])    ⟹  (∑_i) 
                      (Subscript[E, i]) d(Subscript[P, i]) = 
        0
N = (∑_i) (Subscript[P, i]) 
            (Subscript[N, i])    ⟹  (∑_i) 
                      (Subscript[N, i]) d(Subscript[P, i]) = 
        0
\(TextForm\`1 = \(∑\+i P\_i\ \ \ \ \ \ \ \ \ \ \ ⟹\ \ \(∑\+idP\_i\) = 
        0\)\)

Using Lagrange multipliers, we find

\(TextForm\`ln\ P\_i + \ β\ E\_i\  - \ β\ μ\ N\_i\  + \ 
        ln\ 𝒵\  = \ \(0\ \ ⟹\ \ P\_i = \(𝒵\^\(-1\)\) 
          Exp[\(-β\) \((E\_i - μ\ N\_i)\)]\)\)

where, blessed with hindsight, we have chosen to associate Lagrange multipliers β, β μ, and ln 𝒵 with the energy, particle number, and normalization constraints from the outset.

Relationship to canonical ensemble

The microcanonical ensemble stipulates that each state whose energy lies within a specified narrow energy interval occurs with equal probability; the multiplicity Γ[E] = 
        g[E] δ E is used to normalize the probability distribution and hence serves as a microcanonical partition function.  The canonical probability distribution P[
          E] ∝ (e^(-β E)) 
          g[E] dE describes an ensemble composed of a collection of microcanonical ensembles with various energies where each subensemble is weighted by the Boltzmann factor (e^(-β E)).  The canonical partition function which normalizes the probability distribution for this ensemble of ensembles is then a weighted sum of microcanonical partition functions, namely Z = ∫dE g[
              E] (e^(-β E)).  Similarly, the grand canonical ensemble consists of a canonical ensemble with different particle numbers weighted by the relative probability (e^(β μ N)).  It is convenient to define the fugacity parameter as z = (e^(β μ)), whereby the grand partition function becomes

\(TextForm\`𝒵[T, 
        V, μ]\  = \ \(∑\+N\( z\^N\) 
            Z\_N[T, V]\  = \ ∑\+N\( e\^\(β\ μ\ N\)\) Z\_N[T, V]\)\)

where (Subscript[Z, N])[T, V] is the canonical partition function for a system of N particles.  Thus, the factor (z^N) plays a role similar to that of the Boltzmann factor.  The grand partition function is now seen to be a weighted sum of canonical partition functions for subensembles with varying numbers of particles.

The relationship between the canonical and grand canonical ensembles can be clarified further by recognizing that for large N, ln 𝒵 will be dominated by its single largest term, such that

\(TextForm\`0 == \(∂\/∂N\) 
          Log[\(e\^\(β\ μ\ N\)\) 
              Z\_N]\  == \ β\ μ\  + \ \(∂\/∂N\) Log[Z\_N]\ \ \ when\ \ N -> \(N\&~\)\)

However, extensivity of thermodynamic potentials requires Log[(Subscript[Z, N])] ∝ N so that the maximization condition becomes

\(TextForm\`0 == β\ μ\  + \ \(1\/\(N\&~\)\) 
            Log[Z\_\(N\&~\)]\ \ \ ⟹\ \ μ\ \(N\&~\)\  = \ \(-k\_B\) T\ Log[Z\_\(N\&~\)]\)

Now, using \(TraditionalForm\`z = \(e\^\(β\ μ\)\ ⟹\ \ μ = \(k\_B\) T\ Log[z]\)\), we find that the fugacity satisfies

\(TextForm\`\(N\&~\)\ Log[
            z]\  == \ \(-Log[
              Z\_\(N\&~\)]\ \ \ ⟹\ \ Z\_\(N\&~\)\) = 
      z\^\(-\(N\&~\)\)\)

at the peak of the N distribution.  Thus, the most probable particle number is determined by the condition

\(TextForm\`\(μ\ \(N\&~\)\)\/\(\(k\_B\) T\)\  = \ \(\(-Log[
              Z\_\(N\&~\)]\ \ \ ⟹\ \ \ Z\_\(N\&~\)\^\(1/\(N\&~\)\)\) = μ\/\(\(k\_B\) T\)\)\)

Although it may appear that the grand canonical ensemble is more complicated than the other ensembles, sometimes unnecessarily so, it nevertheless plays an important role in a wide variety of applications.  It is often inconvenient to enforce a strict constraint upon particle number or an approximation method may violate particle number conservation, in which case the grand canonical ensemble may provide a more convenient method to adjust the particle number through μ.  Similarly, the grand canonical ensemble provides an easier method for analyzing the effects of indistinguishability among identical particles than does the canonical ensemble.  Furthermore, in many systems the particle number really is indeterminate, such as for photons or phonons.  For other systems, particles may be exchanged between several components of the system, with the equilibrium division of particles governed by the chemical potential.  For example, when a gas is in equilibrium with its own liquid phase, with these two component separated by gravity, the particle numbers in the two phases are statistical variables.  Since the number in the condensed phase is usually much larger, the condensed phase serves as a particle reservoir for the vapor.  The vapor pressure is then governed by the chemical potential established by the reservoir.

Density operators for canonical ensembles

In the energy representation the microcanonical ensemble is represented by the density matrix

\(TextForm\`ρ\_n = Γ\^\(-1\)\ Θ[δ E\  - \ 2 | E - E\_n | ]\)

where the Heaviside or unit step function is defined by Θ[x] = 1 for x > 0, Θ[x] = (1/2) for x = 0, and Θ[x] = 0 for x < 0.  Thus, in an arbitrary representation we could represent the microcanonical density operator in the form

\(TextForm\`ρ\&^ = \(Γ\^\(-1\)\ Θ[δ E\  - \ 2 | 
              E - H\&^ | ]\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Γ\ \  = \ Tr\ Θ[δ E\  - \ 2 | 
              E - H\&^ | ]\)\)

However, this ugly operator is not particularly well suited to calculation, for which one should always return to the energy representation.  Nevertheless, it does provide a formal definition of the ensemble.

The canonical ensemble is represented in the energy representation by the density matrix elements

\(TextForm\`ρ\_n = \(Z\^\(-1\)\ e\^\(\(-β\)\ E\_n\)\ \ \ \ \ \ \ \ \ \ \ \ Z = \ ∑\+n e\^\(\(-β\)\ E\_n\)\)\)

where the summation is with respect to states (not levels).  Thus, in an arbitrary basis we can employ the canonical density operator

\(TextForm\`\(ρ\&^\) = \(Z\^\(-1\)\ e\^\(\(-β\)\ \(H\&^\)\)\ \ \ \ \ \ \ \ \ \ \ \ Z = \ 
        Tr\ e\^\(\(-β\)\ \(H\&^\)\)\)\)

where the exponential is interpreted as a power series.  The ensemble average for any observable is then

\(TextForm\`⟨\(f\&^\)⟩\  = \ \(Tr\ \(ρ\&^\) \(f\&^\)\  = \ \(\(Tr\)\(\ \)\(e\^\(\(-β\)\ \(H\&^\)\)\) \(f\&^\)\(\ \)\)\/\(Tr\ e\^\(\(-β\)\ \(H\&^\)\)\)\)\)

In particular, the internal energy is given by

\(TextForm\`U\  = \ \(⟨\(H\&^\)⟩\  = \ \(\(-\(∂\/∂β\)\) 
            Log[\ Tr\ e\^\(\(-β\)\ \(H\&^\)\)\ ]\  = \ \(\(-\(∂\/∂β\)\) \(Log[
              Z]\)\(\ \)\)\)\)\)

Similarly, the entropy is

\(TextForm\`S\  = \ \(⟨\ \(-k\_B\) 
            Log[\(ρ\&^\)]\ ⟩\  = \ \(\(-k\_B\) 
            Tr \( ρ\&^\) 
            Log[\(ρ\&^\)]\  = \ \(-k\_B\) \(Z\^\(-1\)\) 
            Tr[\(e\^\(\(-β\)\ \(H\&^\)\)\) \((\(-β\)\ \(H\&^\)\  - \ Log[Z])\)\ ]\)\)\)

or

S = (1/T) ( 
          U  +  (Subscript[k, B]) T Log[Z] )

Therefore, we find

\(TextForm\`F = \(U\  - \ 
          T\ S\  = \ \(\(-k\_B\) 
            T\ Log[Z]\ \ ⟹\ \ Z = e\^\(\(-β\)\ F\)\)\)\)

in agreement with our previous results.  Furthermore, we can define a Helmholtz operators as

\(TextForm\`\(F\&^\) = \(\(H\&^\)\  - \ 
          T\ \(S\&^\)\  = \ \(H\&^\)\  + \ \(\(k\_B\) \(T\)\(\ \)\(Log[\(ρ\&^\)]\)\(\ \ \)\)\)\)

so that

\(TextForm\`F = \(Tr\ \(e\^\(\(-β\)\ \(H\&^\)\)\) \(F\&^\)\ \  = \ \(Tr \( ρ\&^\) \(H\&^\)\  - \ 
            T\ Tr \( ρ\&^\) \(S\&^\)\  = \ 
          U\  - \ T\ S\)\)\)

Thus, we can complete the circle by solving for \(TraditionalForm\`\(ρ\&^\)\), with the result

\(TextForm\`\(ρ\&^\)\  = \ \(e\^\(β \((\(F\&^\) - \(H\&^\))\)\)\  = \ e\^\(\(-\(S\&^\)\)/k\_B\)\)\)

Following this pattern, the grand canonical density matrix becomes

\(TextForm\`ρ\_n = \(𝒵\^\(-1\)\ e\^\(\(-β\)\ \((E\_n - μ\ N\_n)\)\)\ \ \ \ \ \ \ \ \ \ \ \ 𝒵 = \ ∑\+n e\^\(\(-β\)\ \((E\_n - μ\ N\_n)\)\)\)\)

in the energy representation.  However, since this ensemble does not have fixed particle number, its states and operators do not live in a unique Hilbert space but must be constructed in a Fock space consisting of the union of all Hilbert spaces of similar nature but different particle numbers.  Hence, in the Fock space there must exist a number operator \(TraditionalForm\`\(N\&^\)\) such that \(TraditionalForm\`\(N\&^\) |n⟩ = \(N\_n\) |n⟩\) for state n where (Subscript[N, n]) is the eigenvalue of the number operator in a representation where both \(TraditionalForm\`\(H\&^\)\) and \(TraditionalForm\`\(N\&^\)\) are diagonal.  The state index n subsumes the entire set of quantum numbers, include particle number, needed to specify a state in Fock space.  The corresponding statistical operators are then

\(TextForm\`\(ρ\&^\) = \(𝒵\^\(-1\)\ e\^\(\(-β\)\ \((\(E\&^\) - μ\ \(N\&^\))\)\)\ \ \ \ \ \ \ \ \ \ \ \ 𝒵 = \ 
        Tr\ e\^\(\(-β\)\ \((\(E\&^\) - μ\ \(N\&^\))\)\)\)\)

such that

\(TextForm\`S\  = \ \(\(-k\_B\) Tr \( ρ\&^\) 
          Log[\(ρ\&^\)]\  = \ \(-k\_B\) 
          Tr[\(ρ\&^\) \((\(-β\)\ \(H\&^\)\  + β\ μ \( 
                        N\&^\) - \ 
                  Log[𝒵])\)\ ]\)\)

or

T S  =  
      U  -  μ N  +  (Subscript[k, B]) 
          T Log[𝒵]

where

\(TextForm\`U = \(Tr\ \(ρ\&^\) \(H\&^\)\ \ \ \ \ \ \ \ \ \ N = 
        Tr\ \(ρ\&^\) \(N\&^\)\)\)

with the traces taken with respect to Fock space, which includes a summation over particle number.  Thus, we find

\(TextForm\`𝒵\  = \ \(Exp[\(-β\) \((U - 
                T\ S\  - \ μ\ N)\)]\  = \ \(e\^\(\(-β\)\ 𝒢\)\ \ \ \ ⟹\ \ 𝒢\  = \ \(-k\_B\) T\ Log[𝒵]\)\)\)

where 𝒢 = U - 
            T S  -  μ N  =  -p V is the grand potential, as before.  Therefore, the canonical density operator can also be expressed as

\(TextForm\`\(\(ρ\&^\)\(\ \)\(=\)\(\ \)\(e\^\(β \((𝒢 - \(H\&^\) + μ\ \(N\&^\))\)\)\)\(\ \)\)\)

Finally, if we define an operator for 𝒢, we find that

\(TextForm\`\(𝒢\&^\) = \(\(H\&^\) - 
          T \( S\&^\) - μ \( 
                  N\&^\)\ \ \ ⟹\ \ \(ρ\&^\) = e\^\(\(-\(S\&^\)\)/k\_B\)\)\)

provides a closed loop of operator definitions.

Therefore, the fundamental definition

\(TextForm\`ρ\&^ = \(e\^\(\(-S\&^\)/k\_B\)\ \ \ ⟹\ \ \ \ \ S\&^\  = \ \(\(-k\_B\) \(Log[ρ\&^]\)\(\ \)\)\)\)

can be regarded as the primary template for the density operators corresponding to various ensembles.  By expressing S in terms of the appropriate thermodynamic variables, replacing those that are variable by operators and treating those that are constrained as numbers, one can construct the desired canonical density matrix practically by inspection.

variable E;  fixed T,V,N        ⟹  \(S\&^\)=\(\(1\/T\) \((H\&^ - 
                    F)\)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ⟹\ \ ρ\&^ = Exp[β \((F - H\&^)\)]\)
\(variable\ E, N; \ fixed\ T, 
      V, μ\ \ \ ⟹\ \ \ S\&^ = \(\(1\/T\) \((H\&^ - μ\ N\&^ - 𝒢)\)\ \ \ ⟹\ \ ρ\&^ = Exp[β \((𝒢\  - H\&^ + μ\ N\&^)\)]\)\)
\(variable\ E, V; \ fixed\ T, p, 
      N\ \ \ ⟹\ \ \(\(\ \)\(S\)\)\&^ = \(\(1\/T\) \((H\&^ + p\ V\&^ - G)\)\ \ \ ⟹\ \ ρ\&^ = 
          Exp[β \((G\  - H\&^ - p\ V\&^)\)]\)\)

The thermodynamic potential whose natural variables are those specified by the external constraints provides the normalization of the density operator.

Problems

\(TextForm\`E\_n = 
      n\  ℏ ω\  + \ \(N\_s\) \(( \(3\/2\) ℏ ω - ϕ)\)\)

\(TextForm\`g[N\_↑, N\_↓, 
        N\_↕]\  ≈ \ 
      2\ \((\(N\_↑!\) \(\(N\_↓\) ! \))\)\/\((\(\((N\_↑ - N\_↕\/2)\)!\) \(\((N\_↓ - N\_↕\/2)\)!\) \((\(\((N\_↕\/2)\)!\))\)\^2)\)\)

H = (({{(Subscript[ε, 0]) - (Subscript[ε, 1]), 0, 0, -2 (Subscript[ε, 1])}, {0, (Subscript[ε, 0]) + (Subscript[ε, 1]), 0, 0}, {0, 0, (Subscript[ε, 0]) + (Subscript[ε, 1]), 0}, {-2 (Subscript[ε, 1]), 0, 0, (Subscript[ε, 0]) - (Subscript[ε, 1])}}))

E = (∑_(i = 1)^N)(-μ B (Subscript[σ, i]) - 
            J (Subscript[σ, i]) (Subscript[σ, i + 1]))

\(TextForm\`Z\  = \ ∑\+{σ\_i}∏\+\(i = 1\)\%N z\_\(i, i + 1\)\)

\(TextForm\`z\_\(i, i + 1\) = 
      Exp[β \((\(μ\ B\)\/2\ \((σ\_i + σ\_\(i + 1\))\) + J\ \(σ\_i\) σ\_\(i + 1\))\)]\)

\(TextForm\`Z = \(∏\+\(k = 1\)\%N Z\_k\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Z\_k = 
        1\/\(1 - e\^\(\(-β\) ε\_k\)\)\)\)

\(TextForm\`\(\(\ \)\(∑\+\(k = 1\)\%N ⋯\ \ ⟶\ \ N\ \(∫\_0\%1 ⋯ dx\ \ \ \ \ with\ \ \ \ x\) = k\/N\)\)\)

\(TextForm\`ε\_n = \((n\_x + n\_y + 
              n\_z)\) ℏ ω\  - \ ε\_B + \(κ\/2\) \((\(V - \(σ\_0\) N\)\/N)\)\^2\)

\(TextForm\`C\_V = \(T \((∂S\/∂T)\)\_\(V, N\) = \(\((∂U\/∂T)\)\_\(V, N\)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ C\_p = \(T \((∂S\/∂T)\)\_\(p, N\) = \((∂H\/∂T)\)\_\(p, N\)\)\)\)\)

\(TextForm\`⟨Δ E\ Δ N⟩\  = \ \(\((∂U\/∂N\&_)\)\_β\) ⟨\((Δ N)\)\^2⟩\)

\(TextForm\`⟨\((Δ N)\)\^2⟩\  = \ \(\(\(N\&_\^2\ \(k\_B\) 
              T\)\/V\) \(κ\_T\)\(\ \ \)\)\)