Statistical Postulate

from Statistical Physics using Mathematica
© James J. Kelly, 1996-2002

Initialization

Defaults and packages

ClearAll["Global`*"];
Off[General::spell, General::spell1]

Needs["Utilities`Notation`"]

Dirac notation

Unprotect @ RowBox;
\(\(RowBox[{"(", 
            TagBox[𝒶𝓇ℊ_, 
              Ket, ℴ𝓅𝓉𝓈___], ")"}] := 
	TagBox[𝒶𝓇ℊ, 
          Ket, ℴ𝓅𝓉𝓈];\)\)
\(\(RowBox[{"(", 
            TagBox[𝒶𝓇ℊ_, 
              Bra, ℴ𝓅𝓉𝓈___], ")"}] := 
	TagBox[𝒶𝓇ℊ, 
          Bra, ℴ𝓅𝓉𝓈];\)\)
\(\(RowBox[{"(", 
            TagBox[𝒶𝓇ℊ_, 
              BraKet, ℴ𝓅𝓉𝓈___], ")"}] := 
	TagBox[𝒶𝓇ℊ, 
          BraKet, ℴ𝓅𝓉𝓈];\)\)
Protect @ RowBox;

Unprotect @ FullForm;
\(\(FullForm /: 
	\ TagBox[
          StyleBox[ℯ𝓍𝓅𝓇_, 
		\ Verbatim[\ ShowSpecialCharacters -> 
                False], 𝓇ℯ𝓈𝓉__], 
          FullForm] := 
	
        TagBox[StyleBox[ℯ𝓍𝓅𝓇, 𝓇ℯ𝓈𝓉], FullForm];\)\)
Protect @ FullForm;

Notation[\(ℋ_\&^\) ⟺ Operator[ℋ_]]

Notation[\(λ_\_ℋ_\&^\) ⟺ \(EigenKet[λ_, ℋ_\&^]\)]

Notation[\[VerticalSeparator]ψ__⟩ ⟺ Ket[ψ__]]
Notation[⟨ψ__\[VerticalSeparator] ⟺ Bra[ψ__]]
Notation[⟨ψf__\[VerticalSeparator]ψi__⟩ ⟺ Braket[{ψf__}, {ψi__}]]

AddInputAlias[\[VerticalSeparator]\[Placeholder]⟩,"ket"]
AddInputAlias[⟨\[Placeholder]\[VerticalSeparator],"bra"]
AddInputAlias[⟨\[Placeholder]\[VerticalSeparator]\[Placeholder]⟩,"braket"]

Ensemble averaging

According to quantum mechanics, the state of an isolated system is represented by a state vector ψ which may be expanded in a complete orthonormal basis according to

ψ = (∑_α) 
              (Subscript[a, α]) (Subscript[ξ, α])

where

(Subscript[a, α])= ⟨(Subscript[ξ, α])|ψ⟩               ⟨(Subscript[ξ, β])|(Subscript[ξ, α])⟩ =(Subscript[δ, α, β])

Suppose that we prepare an ensemble consisting of a very large number of identical systems, all prepared in the same quantum state ψ, and perform the same measurement upon each member of the ensemble.  An observable quantity A is represented by a hermitian operator \(TraditionalForm\`\(\(A\)\(\ \)\)\&^\).  Unless the system is prepared in one of the eigenstates of \(TraditionalForm\`\(\(A\)\(\ \)\)\&^\), these measurements will produce a distribution of results {(Subscript[A, k]),  k = 1, N} whose average value

\(⟨\(A\&^\)⟩\) =⟨ψ| \(A\&^\)|ψ⟩ = (∑_(α, β))(Subscript[a, α])\(a\_β\^*\) ⟨(Subscript[ξ, β])| \(A\&^\)|(Subscript[ξ, α])⟩=\(lim\+\(N -> ∞\)\(1\/N\) \(∑\+\(k = 1\)\%N A\_k\)\)

approaches the quantum mechanical expectation value in the limit of large N.  Even though the state vector obeys a completely deterministic equation of motion

\(TextForm\`i ∂ψ\/∂t = \(H\&^\) ψ\)

the quantum mechanical measurement process is inherently statistical, such that observable quantities require statistical averaging even if the quantum state is known perfectly.

Often it is not possible to prepare an ensemble in which every member is represented by precisely the same quantum state.  For example, suppose that a polarized ion source is used to produce a beam of protons.  The source may be designed to produce more protons with spin up than with spin down, but it is usually not possible to guarantee that every proton has spin up.  Therefore, we consider a statistical ensemble consisting of many similar, but necessarily identical, systems and let (Subscript[P, i]) represent the probability, or relative frequency, with which a state (Subscript[ψ, i]) appears within the ensemble.  If our polarized source is very good, (Subscript[P, ↑]) should be close to unity and (Subscript[P, ↓]) should be small.  Each state (Subscript[ψ, i]) in the ensemble can be expanded in a complete orthonormal basis according to

(Subscript[ψ, i]) = (∑_α) 
              (Subscript[a, i, α]) (Subscript[ξ, α])

To obtain the ensemble average of the expectation value of \(TraditionalForm\`\(A\&^\)\), we must weight the expectation value ⟨(Subscript[A, i])⟩ with the frequency (Subscript[P, i]) for the corresponding wave function (Subscript[ψ, i]).  Hence, the ensemble average becomes

\(\(\(⟨\(A\&^\)⟩\)\(\ \)\)\&_\)=(∑_i) (Subscript[P, i]) (Subscript[A, i])=(∑_i)(Subscript[P, i])⟨(Subscript[ψ, i])|\(A\&^\)|(Subscript[ψ, i])⟩  = (∑_i)(∑_(α, β))(Subscript[P, i])(Subscript[a, i, α])\(a\_\(i, β\)\^*\)⟨(Subscript[ξ, β])|\(A\&^\)|(Subscript[ξ, α])⟩

where the overline indicates ensemble averaging with respect to states and angle brackets denote quantum-mechanical averaging.  Notice that \(TraditionalForm\`⟨ξ\_β | 
          A\&^ | ξ\_α⟩\) is independent of the properties of the ensemble.  Therefore, it is useful to define a density operator

\(ρ\&^\) = (∑_i)|(Subscript[ψ, i])⟩(Subscript[P, i])⟨(Subscript[ψ, i])|

with matrix elements

(Subscript[ρ, α, β]) =⟨(Subscript[ξ, α])|\(ρ\&^\)|(Subscript[ξ, β])⟩ = (∑_i)⟨(Subscript[ξ, α])|(Subscript[ψ, i])⟩(Subscript[P, i])⟨(Subscript[ψ, i])|(Subscript[ξ, β])⟩=\(∑\+i\(\( 
                    P\_i\) \(a\_\(i, α\)\) \(a\_\(i, β\)\^*\)\(\ \)\)\)

that describes the statistical properties of the ensemble.  Recognizing that the probabilities (Subscript[P, i]) are real, we observe that the density operator is hermitian because the matrix elements satisfy

\(TextForm\`ρ\_\(β, α\)\  = \ \(\(ρ\_\(α, β\)\^*\)\ \ \ \ \ ⟹\ \ \ \ \ \(ρ\&^\)\^† = \(ρ\&^\)\)\)

The ensemble average for the expectation value of \(TraditionalForm\`A\&^\) can now be expressed in the form

\(⟨\(A\&^\)⟩\&_\)  = (∑_i)(Subscript[P, i])⟨(Subscript[ψ, i])|\(A\&^\)|(Subscript[ψ, i])⟩ = \(Tr\ ρ\&^ A\&^\)

because

\(Tr\ ρ\&^ A\&^\) =(∑_α)⟨(Subscript[ξ, α])|\(ρ\&^\) \(A\&^\)|(Subscript[ξ, α])⟩=(∑_(α, β))⟨(Subscript[ξ, α])|\(ρ\&^\)|(Subscript[ξ, β])⟩⟨(Subscript[ξ, β])|\(A\&^\)|(Subscript[ξ, α])⟩ =(∑_(i, α, β))(Subscript[P, i])(Subscript[a, i, α])\(a\_\(i, β\)\^*\) ⟨(Subscript[ξ, β])|\(A\&^\)|(Subscript[ξ, α])⟩

Thus, the statistical matrix (Subscript[ρ, α, β]) facilitates ensemble averaging of quantum mechanical expectation values.

The density operator can be expressed in an arbitrary representation as

\(ρ\&^\) = (∑_(α, β))|(Subscript[ξ, α])⟩(Subscript[ρ, α, β]) ⟨(Subscript[ξ, β])|

where

(Subscript[ρ, α, β]) = ⟨(Subscript[ξ, α])|\(ρ\&^\)|(Subscript[ξ, β])⟩\(\(=\)\(ρ\_\(β, α\)\^*\)\)

are matrix elements with respect to the basis denoted by |(Subscript[ξ, α])⟩.  The diagonal matrix elements (Subscript[ρ, α, α]) represent the probability for state |(Subscript[ξ, α])⟩, whereas the magnitudes of off-diagonal matrix elements |(Subscript[ρ, α, β])| represent the probabilities for spontaneous transitions between states |(Subscript[ξ, α])⟩ and |(Subscript[ξ, β])⟩.  The symmetry \(TraditionalForm\`\(\(|\)\(ρ\_\(α, β\)\)\(|\)\) = \(\(|\)\(ρ\_\(β, α\)\)\(|\)\)\) is known as the principle of detailed balance, which states that transition probabilities for both forward and backward processes, α ↔ β, are equal.  Detailed balance is a consequence of time-reversal invariance of the hamiltonian.

Consider another basis, denoted by |(Subscript[ϕ, α])⟩, obtained by a unitary transformation U satisfying \(TraditionalForm\`U U\^† = 𝕀\) where 𝕀 is the unit matrix.  These bases are related by

\(\(\(|\)\(ξ\_α\)\)\(⟩\)\) = (∑_β)(Subscript[U, α, β])|(Subscript[ϕ, β])⟩  ,            \(\(\(|\)\(ϕ\_β\)\)\(⟩\)\) = (∑_γ)(Subscript[U, β, γ]^†)|(Subscript[ξ, γ])⟩

Matrix elements of the density operator in these two representations are then related by

⟨(Subscript[ϕ, α])|\(ρ\&^\)|(Subscript[ϕ, β])⟩ = (∑_((α^′), (β^′)))⟨(Subscript[ξ, (α^′)])|(Subscript[U, (α^′), α])\(ρ\&^\)(Subscript[U, β, (β^′)]^†)|(Subscript[ξ, (β^′)])⟩

Therefore, the density operator transforms according to

\(TextForm\`ξ⟶
        U\ ϕ\ \ \ \ \ \ ⟹\ \ \ \ ρ\ ⟶\ U\ ρ\ U\^†\)

under a unitary transformation of the basis states.  Nevertheless, because the trace of any operator is independent of basis, the equation \(TraditionalForm\`⟨\(A\&^\)⟩\&_ = 
        Tr\ \ ρ\&^ A\&^\) can be used in any basis, diagonal or not.  To verify that the trace is independent of basis, we note that

\(TextForm\`Tr\ B\&^ A\&^\  = \ \(∑\+\(i, j\)\(B\_\(i, j\)\) 
            A\_\(j, i\) = \ \(∑\+\(i, j\)\(A\_\(i, j\)\) 
              B\_\(j, i\)\  = \ \(Tr\ A\&^ B\&^\ \ \ ⟹\ \ \ Tr\ U \ A\&^\  U\^† = \(Tr\ A\&^\  \(U\^†\) U\  = \ 
              Tr\ A\&^\)\)\)\)\)

for any unitary transformation satisfying  (U^†) 
          U = 𝕀.  Furthermore, because a hermitian matrix can be diagonalized by means of a unitary transformation, there exists at least one representation in which the density operator is diagonal.  Therefore, in a diagonal representation represented by the complete orthonormal basis |(Subscript[η, α])⟩, the density operator takes the simple form

\(ρ\&^\) = (∑_α) |(Subscript[η, α])⟩ (Subscript[P, α]) ⟨(Subscript[η, α])|

where the coefficients (Subscript[P, α]) are real eigenvalues satisfying

\(ρ\&^\)|(Subscript[η, α])⟩ =  (Subscript[P, α])|(Subscript[η, α])⟩

From the original definition of the density operator it is clear that (Subscript[P, α]) ≥ 0 is positive definite and represents the frequency with which state |(Subscript[η, α])⟩ appears in the ensemble.

Although it is often difficult to construct a diagonal representation explicitly, the mere existence of such a representation greatly simplifies formal derivations.  Furthermore, it is often inconvenient, particularly in numerical simulations, to enforce the normalization of the density operator.  Under those circumstances we express the ensemble average in the form

\(TextForm\`⟨\(A\&^\)⟩\&_\ \  = \ \(Tr\ ρ\&^ A\&^\)\/\(Tr\ ρ\&^\)\)

where the denominator renormalizes the density operator.  We will soon find that the density matrix provides a convenient and powerful method for studying the statistical properties of quantum ensembles and is easily generalized to include classical ensembles also.  Beyond thermodynamics, density-matrix methods find widespread applications to many topic, including scattering theory, beam transport, and measurement theory.

Pure and mixed ensembles

Suppose that we could prepare a pure ensemble in which every member is represented by the same wave function, ψ.  If we happen to choose a representation in which that state is one of the basis states, we would find that one of the diagonal matrix elements of the density operator would be unity and that all other matrix elements would vanish.  However, if we are working with a representation in which

ψ = (∑_α) 
              (Subscript[a, α]) (Subscript[ξ, α])

appears  to be a nontrivial linear superposition of basis states, the density matrix would appear to be much more complicated and it might appear, at first glance, that the ensemble contains a mixture of states.  Nevertheless, one can distinguish a pure ensemble from a mixed ensemble by means of a simple statistical test.  The ensemble average of any operator \(TraditionalForm\`\(A\&^\)\) can be evaluated according to

\(⟨\(A\&^\)⟩\&_\)  = \(Tr\ ρ\&^ \(\(A\)\(\ \)\)\&^\) =  (∑_i)(Subscript[P, i])⟨(Subscript[ψ, i])|\(A\&^\)|(Subscript[ψ, i])⟩

and is simplest in a diagonal representation.  Consider the ensemble average of the unit operator.  If we let \(TraditionalForm\`A\&^ -> 𝕀\&^\), we find \(TraditionalForm\`Tr\ ρ\&^\  = \ \(∑\+i P\_i = 1\)\) for a properly normalized ensemble.  On the other hand, if we let \(TraditionalForm\`A\&^ -> ρ\&^\), we find

\(TextForm\`Tr\ ρ\&^\^2\  = \ ∑\+i P\_i\^2\  ≤ \ 
        Tr\ ρ\&^\)

where equality pertains only for a pure ensemble for which there exists a representation in which only a single (Subscript[P, i]) is nonzero.  More general ensembles composed of more than one unique stationary state are known as mixed ensembles andcannot be reduced to a single state vector in any representation.  Therefore, we the rules

\(TextForm\`		ρ\&^\^2 = ρ\&^\ \ \ \ \ \ \ \ \ \ \ \ \ \ ⟹\ \ pure\ ensemble\)
\(TextForm\`		Tr\ ρ\&^\^2 < 
      Tr\ ρ\&^\ \ \ ⟹\ \ mixed\ ensemble\)

distinguish between pure and mixed ensembles independently of representation.

Unfortunately, one commonly finds pure or mixed ensembles described as pure or mixed states.  This terminology is misleading because any state can be represented as a linear superposition of basis states, and hence seems to be mixed, but an ensemble formed from a single state vector is pure no matter how that state vector is distributed among the basis states.  The difference is that a mixed ensemble can never be formed from a single state vector, no matter how complex its representation, while any pure ensemble contains only a single state vector no matter how complex its density matrix.

Example: density matrix for spin (1/2)

Consider a pure state described by the Pauli spinor

ψ=(({{a}, {b}}))

whose density matrix is

\(ρ\&^\) = |ψ⟩⟨ψ| = (({{a}, {b}}))(({{\(a\^*\), \(b\^*\)}})) = (({{\(a\ \(a\^*\)\), \(a\ \(b\^*\)\)}, {\(b\ \(a\^*\)\), \(b\ \(b\^*\)\)}}))

The polarization \(TraditionalForm\`P\&⇀ = 
        Tr\ \(ρ\&^\) σ\&⇀\) is the expectation value of the spin operator, where

(Subscript[σ, x])=(({{0, 1}, {1, 0}}))                 (Subscript[σ, y])=(({{0, -i}, {i, 0}}))              (Subscript[σ, z])=(({{1, 0}, {0, -1}}))

in the Pauli representation.  Direct computation yields

⟨ψ|(Subscript[σ, x])|ψ⟩ = (({{\(a\^*\), \(b\^*\)}}))(({{0, 1}, {1, 0}}))(({{a}, {b}})) = \(b\ \(a\^*\)\  + \ a\ \(b\^*\)\)
⟨ψ|(Subscript[σ, y])|ψ⟩ = (({{\(a\^*\), \(b\^*\)}}))(({{0, -i}, {i, 0}}))(({{a}, {b}})) = \(\(-i\) \((b\ \(a\^*\)\  - \ a\ \(b\^*\))\)\)
 ⟨ψ|(Subscript[σ, z])|ψ⟩ = (({{\(a\^*\), \(b\^*\)}}))(({{1, 0}, {0, -1}}))(({{a}, {b}})) = \(a\ \(a\^*\)\  - \ b\ \(b\^*\)\)

so that

(Overscript[P, ⇀]) = (({{\(\(2\)  \(Re\) \((b\ \(a\^*\))\)\(\ \)\)}, {\(2  Im \((b\ \(a\^*\))\)\)}, {\(|a|\^2\  - \ |b|\^2\)}}))

is the polarization for a pure state.  Alternatively, we could employ the trace method, whereby

(Subscript[P, x])=\(Tr\ ρ\&^ σ\_x\)=Tr(({{\(a\ \(a\^*\)\), \(a\ \(b\^*\)\)}, {\(b\ \(a\^*\)\), \(b\ \(b\^*\)\)}}))(({{0, 1}, {1, 0}}))=Tr(({{\(a\ \(b\^*\)\), \(a\ \(a\^*\)\)}, {\(b\ \(b\^*\)\), \(b\ \(a\^*\)\)}}))= \(b\ \(a\^*\)\  + \ a\ \(b\^*\)\)
(Subscript[P, y])=\(Tr\ ρ\&^ σ\_y\)=Tr(({{\(a\ \(a\^*\)\), \(a\ \(b\^*\)\)}, {\(b\ \(a\^*\)\), \(b\ \(b\^*\)\)}}))(({{0, -i}, {i, 0}}))=Tr(({{\(i\ a\ \(b\^*\)\), \(\(-i\)\ b\ \(a\^*\)\)}, {\(i\ b\ \(b\^*\)\), \(\(-i\)\ b\ \(a\^*\)\)}}))=\(\(-i\) \((b\ \(a\^*\)\  - \ 
                    a\ \(b\^*\))\)\)
(Subscript[P, z])=\(Tr\ ρ\&^ σ\_z\)=Tr(({{\(a\ \(a\^*\)\), \(a\ \(b\^*\)\)}, {\(b\ \(a\^*\)\), \(b\ \(b\^*\)\)}}))(({{1, 0}, {0, -1}}))=Tr(({{\(a\ \(b\^*\)\), \(\(-a\)\ \(b\^*\)\)}, {\(b\ \(a\^*\)\), \(\(-b\)\ \(b\^*\)\)}}))= \(a\ \(a\^*\)\  - \ b\ \(b\^*\)\)

in agreement with the previous result.

The polarization vector for any pure state must have unit magnitude because the particle definitely has spin (1/2).

\(TextForm\`\(\(|\)\(P\&⇀\)\( | \^2\)\) = \(\(\(|\)\(\ \)\(b\ \(a\^*\)\  + \ 
          a\ \(b\^*\)\)\( | \^2\)\(\ \)\(+\ \(\(|\)\(b\ \(a\^*\)\  - \ 
              a\ \(b\^*\)\)\( | \^2\)\(\ \)\(+\ \(\(|\)\(a\ \(a\^*\)\  - \ 
                  b\ \(b\^*\)\)\( | \^2\)\)\)\)\)\)\ 

      \(\(=\)\(\ \)\(2 | a\( | \^2\) | 
          b\( | \^2\)\(+2\) Re \((a\ \(b\^*\)\ b\ \(a\^*\))\)\  + \ 2 | 
          a\( | \^2\) | 
          b\( | \^2\)\(\(\(-2\) 
              Re \((a\ \(b\^*\) b\ \(a\^*\))\)\)\(\ \)\(+\)\)\  | 
          a\( | \^4\)\(+\(\(|\)\(b\)\( | \^4\)\(-2\)\(|\)\(a\)\( | \^2\)\(|\)\(b\)\( | \^2\)\)\)

        \(\(=\)\(\(\(|\)\(a\)\( | \^4\)\(+\(\(|\)\(b\)\( | \^4\)\(+2\)\(|\)\(a\)\( | \^2\)\(|\)\(b\)\( | \^2\)\)\)\)

          \(\(=\)\(\((\(|\)\(a\)\( | \^2\)\(+\(\(|\)\(b\)\( | \^2\)\)\))\)\^2

            \(\(=\)\(1\)\)\)\)\)\)\)\)\)\)

Therefore, any pure state is completely polarized in some direction.  A few examples follow.

({{ψ, (Overscript[P, ⇀]), \(TraditionalForm\`ρ\&^\), spin direction}, {(({{1}, {0}})), (({{0}, {0}, {1}})), (({{1, 0}, {0, 0}})), \(+z\&^\)}, {(({{0}, {1}})), (({{0}, {0}, {-1}})), (({{0, 0}, {0, 1}})), \(-z\&^\)}, {(1/(Sqrt[2]))(({{1}, {1}})), (({{1}, {0}, {0}})), (({{(1/2), (1/2)}, {(1/2), (1/2)}})), \(+x\&^\)}, {(1/(Sqrt[2]))(({{1}, {-1}})), (({{-1}, {0}, {0}})), (({{(1/2), -(1/2)}, {-(1/2), (1/2)}})), \(-x\&^\)}, {(1/(Sqrt[2]))(({{1}, {i}})), (({{0}, {1}, {0}})), (({{(1/2), -(i/2)}, {(i/2), (1/2)}})), \(+y\&^\)}, {(1/(Sqrt[2]))(({{1}, {-i}})), (({{0}, {-1}, {0}})), (({{(1/2), (i/2)}, {-(i/2), (1/2)}})), \(-y\&^\)}})

Any other direction can be obtained using an appropriate linear combination of basis states.

Given that the space of 2×2 matrices is spanned by the set of matrices \(TraditionalForm\`{𝕀, σ\&⇀}\), an arbitrary density matrix can be represented in the form

ρ = (1/2) (𝕀 + 
                (Overscript[P, ⇀])·(Overscript[σ, ⇀])) = (1/2)(({{1 + 
                      (Subscript[P, z]), (Subscript[P, x]) - i (Subscript[P, y])}, {(Subscript[P, x]) + i (Subscript[P, y]), 1 - 
                      (Subscript[P, z])}}))

such that Tr ρ = 1 as required by the normalization of the wave function.  Recognizing that (Overscript[σ, ⇀]) is hermitian, hermiticity of ρ requires that (Overscript[P, ⇀]) be real.  Also note that Tr (Subscript[σ, i]) = 0.  To determine the requirements for a pure ensemble, we evaluate

\(TextForm\`ρ\^2 = \(\(1\/4\) \((1 + 
              2  P\&⇀·σ\&⇀ + \((P\&⇀·σ\&⇀)\)\^2)\)\  = \(1\/4\) \((1 + 2  P\&⇀·σ\&⇀ +  | 
              P\&⇀\( | \^2\))\)\)\)

so that

\(TextForm\`Tr\ ρ\^2 = \(1\/2\) \((\(\(1\)\(+\)\) | 
            P\&⇀\( | \^2\))\)\)

Therefore, a pure ensemble requires \(TraditionalForm\`\(\(|\)\(P\&⇀\)\( | \^2\)\) = 1\), as expected, whereas Tr (ρ^2) < 1 for a mixed ensemble requires \(TraditionalForm\`\(\(|\)\(P\&⇀\)\( | \^2\)\(\(<\)\(1\)\)\)\).  Hence, (Overscript[P, ⇀]) must be a real vector with \(TraditionalForm\`\(\(|\)\(P\&⇀\)\(|\)\(\(≤\)\(1\)\)\)\).  Finally, upon evaluation of the ensemble average

\(TextForm\`⟨σ\&⇀⟩ = \(Tr\ ρ σ\&⇀\  = \ \(\(1\/2\) 
            Tr \((σ\&⇀ + 
                P\&⇀·σ\&⇀  σ\&⇀)\)\  = \ P\&⇀\)\)\)

we conclude that (Overscript[P, ⇀]) represents the average polarization for an ensemble.  Clearly, |(Overscript[P, ⇀])| = 0 corresponds to an unpolarized ensemble while density matrices with |(Overscript[P, ⇀])|<1 describe mixed ensembles with partial polarization.

Ordinarily, one visualizes an unpolarized beam as a stream of particles whose spin orientations are random, equally likely to be found in any direction, but we have found that an unpolarized beam is described by a density matrix that is simply proportional to the unit matrix, such that \(TraditionalForm\`P\&⇀ = \(0\  ⇒ \ ρ\&^ = \(1\/2\) 𝕀\)\) for spin (1/2).  More generally, such a beam can be composed of equal admixtures of oppositely polarized beams because the density matrices for \(TraditionalForm\`\(±z\&^\)\), \(TraditionalForm\`\(±y\&^\)\), or \(TraditionalForm\`\(±x\&^\)\) add to the unit matrix.  However, such beams may not be constructed simply by adding the corresponding spinors because a unique spinor implies unit polarization.  For example, superposing spin up and spin down with positive phase

(1/(Sqrt[2]))[(({{1}, {0}})) + (({{0}, {1}}))] = (1/((Sqrt[2])))(({{1}, {1}}))

yields a pure state polarized in the \(TraditionalForm\`x\&^\) direction.  The difference between adding density matrices and adding spinors is found in the phase relationships among the basis states — the phases for members of an ensemble are random and unrelated.  Suppose that we add two spinors with relative phase δ, such that

ψ=(1/(Sqrt[2]))(({{1}, {(e^(i δ))}}))  ⟹  ρ[δ]=(1/2)(({{1, \(e\^\(\(-i\) δ\)\)}, {\(e\^\(i δ\)\), 1}}))  ⟹  \(ρ\&_\) = (1/2)(({{1, 0}, {0, 1}}))

is the corresponding density matrix.  If the phase is random, varying rapidly in time or between different members of the ensemble, the off-diagonal elements, \(TraditionalForm\`e\^\(\(±i\) δ\)\), must average to zero so that \(TraditionalForm\`ρ\&^ -> \(1\/2\) 𝕀\) for an unpolarized beam.  More generally, we can represent an arbitrary spinor using two real amplitudes a, b and a relative phase δ

ψ=(({{a}, {b (e^(i δ))}}))  ⟹  ρ[δ]=(({{(a^2), \(a b e\^\(\(-i\) δ\)\)}, {\(a b e\^\(i δ\)\), (b^2)}}))  ⟹  \(ρ\&_\) = (({{(a^2), 0 }, {0 , (b^2)}}))

such that the density matrix is diagonal whenever the phases are random.  In most situations, such as a beam of particles, it is not possible to maintain coherence between members of an ensemble.  Therefore, one normally invokes the postulate of random phases and employs a diagonal density matrix comprised of relative probabilities for each basis state.

Stationary ensembles

Thermodynamic equilibrium requires the macroscropic properties of a system to be independent of time.  From a quantum mechanical point of view, thermodynamic equilibrium requires the ensemble average

\(TextForm\`⟨\(A\&^\)⟩\&_\ \  = \ \(Tr\ ρ\&^ A\&^\)\/\(Tr\ ρ\&^\)\)

for expectation values of the operators representing those macroscopic properties to be independent of time.  Assuming that the operator \(TraditionalForm\`\(A\&^\)\) carries no explicit time dependence, a stationary observable then requires the density matrix to be independent of time also.  The time dependence of \(TraditionalForm\`\(ρ\&^\)\) can be obtained most easily using a diagonal representation, here simply denoted by |i⟩.  Using the Schrödinger equation, we find

\(i\ dρ\&^\/dt\)=(∑_i)[((Subscript[P, i])(i (d/d)t|i⟩)⟨i| + |i⟩(i (d/d)t⟨i| )) + i ∂((Subscript[P, i])/∂)t|i⟩⟨i| ]
=(∑_i)[(Subscript[P, i])(\(H\&^\)|i⟩⟨i|  - |i⟩⟨i| \(H\&^\)) + i ∂((Subscript[P, i])/∂)t|i⟩⟨i| ]

such that

\(TextForm\`dρ\&^\/dt = \ \(-i\ [\(H\&^\), ρ\&^]\)\  + \ ∂ρ\&^\/∂t\)

The first term describes the evolution of the ensemble due to the evolution of the state vectors themselves and, hence, is governed by the hamiltonian of the system.  The second term describes the possible time dependence of the state probabilities (Subscript[P, i])[t] due to changing conditions of the environment.  However, because we usually maintain the environment in a steady state, or with very slow variation, the second term is usually negligible.  Furthermore, we assume that the hamiltonian does not depend explicitly on time so that the energy of each system is conserved (in a macroscopic sense).  Thus, in the energy representation we find

\(TextForm\`ρ\&^ = \(ρ[
            H\&^]\ \ ⟹\ \ \([H\&^, ρ\&^]\) = \(0\ \ ⟹\ \ \(dρ\&^\/dt\) = ∂ρ\&^\/∂t -> 0\)\)\)

Therefore, equilibrium is represented by a stationary ensemble whose density matrix is an explicit function of a time-independent hamiltonian.

The canonical ensemble, familiar from elementary statistical mechanics, stipulates a density operator of the form

\(TextForm\`ρ\&^\  = \ e\^\(\(-β\)\ H\&^\)\)

This density operator satisfies the conditions required for a stationary ensemble — it is an explicit function of \(TraditionalForm\`\(H\&^\)\) and, hence, is stationary if \(TraditionalForm\`\(H\&^\)\) has no explicit time dependence.  Using an energy representation, ensemble averages then take the familiar form

\(TextForm\`⟨\(A\&^\)⟩\&_\ \  = \ \ \(\(Tr\ ρ\&^ A\&^\)\/\(Tr\ ρ\&^\)\  = \ \(∑\+i\( e\^\(\(-β\)\ E\_i\)\) A\_i\)\/\(∑\+i e\^\(\(-β\)\ E\_i\)\)\)\)

in which states are weighted by the Boltzmann factor (e^(-β (Subscript[E, i]))).  Thus, the canonical ensemble is based upon a particularly useful example of a stationary density operator, but we will find that other density operators are also useful.

Disorder

Consider an experiment consisting of N independent random trials with n possible outcomes for each trial.  For example, there are n = 6 possible outcomes for each cast of a standard die.  In the absence of information to the contrary, we assign equal a priori probabilities to each possible outcome, such that {(Subscript[P, i]) = (n^(-1)),  i = 1, n}.  Thus, a uniform probability distribution represents the situation in which we possess the least information about the system: we know only the number of possible states of the die.  From the probability distribution {(Subscript[P, i])} we can use standard combinatorial techniques to compute the probability for any particular outcome of the N trials.

Alternatively, we can determine objective probabilities by measuring the frequency (Subscript[f, i]) = (Subscript[N, i])/N for each possible outcome.  In the limit of many trials, the frequency approaches a limiting value identified as the objective probability

(Subscript[P, i]) = 
      (lim_(N -> ∞))((Subscript[N, i])/N)

If, for example, we discovered that (Subscript[P, 1]) = 1 and {(Subscript[P, i]) = 0,  i = 2, n}, we could then predict the outcome of subsequent trials with nearly complete certainty.  Our information about the system is maximized when the probability distribution exhibits the least disorder (it is the most asymmetric).  We describe an unbiased die as fair in part because no one can possess more information about its properties than we, whereas a biased die is unfair to the gambler who lacks the information that is potentially available.  Intermediate situations in which the predictability of the outcome for each trial is neither completely certain nor completely uncertain are characterized by probability distributions which are neither completely concentrated nor completely uniform.  Hence, we seek an objective measure of the disorder D within a probability (or frequency) distribution with the following properties.

It can be shown that these requirements determine the disorder function

D  =  -(∑_(i = 1)^n) (Subscript[P, i]) 
            ln (Subscript[P, i])

uniquely, up to an arbitrary multiplicative scale factor chosen here to be unity.  Although we will not provide the uniqueness proof, it is worth demonstrating that the proposed definition for disorder satisfies each of the requirements above.

Obviously, D vanishes if any (Subscript[P, i]) = 1.  Furthermore, we can use the method of Lagrange multipliers to prove that D is maximized for a uniform probability distribution.  At an extremum of D infinitesimal variations δ (Subscript[P, i]) in the probabilities should produce no first-order change δ D, such that

\(TextForm\`δ D = \(∑\+\(i = 1\)\%n\(∂D\/∂P\_i\) δ P\_i = \(0\ \ ⟹\ \ \(∑\+\(i = 1\)\%n\((1 + 
                    ln P\_i)\) δ P\_i\) = 0\)\)\)

The constraint

\(TextForm\`∑\+\(i = 1\)\%n P\_i = \(1\ \ ⟹\ \ \(∑\+\(i = 1\)\%n δ P\_i\) = 
        0\)\)

is enforced using a Lagrange multiplier λ, such that

\(TextForm\`\(\(\ \ \)\(∑\+\(i = 1\)\%n\((1 + 
                ln P\_i)\) δ P\_i\  + \ λ \(\(\(\ \)\(∑\)\)\+\(i = 
                1\)\%n\) δ P\_i = 
      0\)\)\)

where the variations δ (Subscript[P, i]) can now be treated independently, so that

\(TextForm\`1\  + \ 
        ln P\_i\  + \ λ\  = \ \(0\ \ ⟹\ \ ln P\_i = \(-\((1 + λ)\)\)\ \ for\ all\ i\)\)

Thus, we obtain maximum disorder for a uniform probability distribution:

{(Subscript[P, i]) = (n^(-1)),  i = 1, 
          n}  ⟹  D = (Subscript[D, max]) = 
        ln n

Finally, if

(Subscript[D, 1]) = - (∑_(i = 1)^n) (Subscript[p, i]) 
            ln (Subscript[p, i])
(Subscript[D, 2]) = - (∑_(j = 1)^m) (Subscript[q, j]) 
            ln (Subscript[q, j])

are partial disorders for two independent experiments with probability distributions {(Subscript[p, i]),  i = 1, n} and {(Subscript[q, j]),  j = 1, m}, the composite disorder within the combined experiment would be

\(TextForm\`D = \(\(-\ \(∑\+\(i, j\)\(p\_i\) \(q\_j\) 
              ln \((\(p\_i\) 
                  q\_j)\)\)\)\ \  = \ \(-\ \(∑\+\(i, j\)\((\ \(p\_i\) \((\(q\_j\) ln\ q\_j)\) + \(q\_j\) \((\(p\_i\) 
                      ln\ p\_i)\)\ )\)\)\)\)\)

assuming statistical independence, (Subscript[P, i, j]) = (Subscript[p, i]) (Subscript[q, j]), applies.  Hence, using (∑_i) (Subscript[p, i]) = 1 and (∑_j) (Subscript[q, j]) = 1, we find

D = (Subscript[D, 1]) + (Subscript[D, 2])

and conclude that disorder is additive for statistically independent systems.

Therefore, the function D = -(∑_i) (Subscript[P, i]) ln (Subscript[P, i]) satisfies all the characteristics required to measure disorder.  It is also useful to observe that D can be interpreted as an ensemble average of ln (Subscript[P, i]), whereby

\(TextForm\`D\  = \ \(⟨\(-ln\)\ P⟩\  = \ \(-\(∑\+i\( P\_i\) ln\ P\_i\)\)\)\)

Of course, (Subscript[P, i]) is itself defined as the frequency with which state i occurs in the ensemble.

Correlations Reduce Disorder

Suppose that two systems interact and let (Subscript[P, i, j]) represent the probability that system 1 is found in state i while system 2 is simultaneously found in state j, such that (∑_(i, j))(Subscript[P, i, j]) = 1.  Further, let (Subscript[P, i]^((1))) be the probability that system 1 is found in state i independent of the state of system 2, while (Subscript[P, j]^((2))) is the probability for state j of system 2 independent of system 1, such that

\(TextForm\`P\_i\%\((1)\) = \ \(∑\+j P\_\(i, j\)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ P\_j\%\((2)\) = \ ∑\+i P\_\(i, j\)\)\)
(∑_i) (Subscript[P, i]^((1))) =  (∑_j) 
          (Subscript[P, j]^((2)))  =  (∑_(i, j))(Subscript[P, i, j])  =  
          1

The combined disorder is

D  =  - (∑_(i, j))(Subscript[P, i, j]) 
            ln (Subscript[P, i, j])

whereas the individual disorders would be

\(TextForm\`D\_1\  = \ \(\(-\ \(∑\+i\( P\_i\%\((1)\)\) 
              ln\ P\_i\%\((1)\)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ D\_2\)\)\  = \ \(-\ \(∑\+j\( P\_j\%\((2)\)\) ln\ P\_j\%\((2)\)\)\)\)\)

if the two systems were isolated and had the same distributions as when interacting.  To compare these disorders, we note that

D  -  (Subscript[D, 1])  -  
        (Subscript[D, 2])  =  (∑_i) (Subscript[P, i]^((1))) 
            ln (Subscript[P, i]^((1)))  +  (∑_j) (Subscript[P, j]^((2))) 
            ln (Subscript[P, j]^((2)))  -  (∑_(i, j))(Subscript[P, i, j]) 
            ln (Subscript[P, i, j])
= (∑_(i, j))( (Subscript[P, i, j]) 
              ln (Subscript[P, i]^((1)))  +  (Subscript[P, i, j]) 
              ln (Subscript[P, j]^((2)))  -  (Subscript[P, i, j]) 
              ln (Subscript[P, i, j]) )

such that

D - (Subscript[D, 1]) - 
        (Subscript[D, 2]) = (∑_(i, j))(Subscript[P, i, 
              j]) ln (((Subscript[P, i]^((1))) 
                (Subscript[P, j]^((2))))/( (Subscript[P, i, 
                  j])))

Recognizing that

\(TextForm\`ln\ x ≤ 
        x - 1\ \ ⟹\ \ \(∑\+\(i, j\)\(P\_\(i, j\)\) 
                ln\ \(\(P\_i\%\((1)\)\) P\_j\%\((2)\)\)\/\(\(\ \)\(P\_\(i, j\)\)\)\) ≤ \ ∑\+\(i, j\)\((\(P\_i\%\((1)\)\) P\_j\%\((2)\) - 
              P\_\(i, j\))\) = 0\)

we discover that

D ≤ (Subscript[D, 1]) + (Subscript[D, 2])

where equality requires statistical independence ((Subscript[P, i, j]) = (Subscript[P, i]^((1))) (Subscript[P, j]^((2)))).  Therefore, correlations between subsystems reduce the composite disorder; disorder is greatest for independent, noninteracting subsystems.

For many systems considerable insight can be obtained using an ideal approximation that neglects interactions between elements or subsystems.  For example, the ideal approximation treats a fluid as a collection of noninteracting particles within a container distributed among single-particle energy levels; this is a good approximation for a low-density gas and remains useful even as the condensation phase transition is approached.  Similarly, ideal paramagnetism neglects spin-spin interactions between atoms; this is a good approximation for paramagnetic salts in which only a relatively small fraction of the atoms carry magnetic moments and the separation between magnetically active atoms is large.  For other systems we might be able to determine the spectrum of normal modes of vibration or excitation and to treat these normal modes as statistically independent subsystems.  Therefore, ideal systems consisting of N noninteracting, statistically independent elements or subsystems represent an important limiting case.  We now demonstrate that additivity of disorder for statistically independent subsystems can be generalized to N elements.

Let (Subscript[σ, j]) represent the quantum numbers needed to specify the state of element j.  The probability (Subscript[P, N])[{(Subscript[σ, j])}] for a particular configuration of N statistically independent elements, represented by the set of state variables {(Subscript[σ, j]),  j = 1, N}, is simply the product of the probabilities (Subscript[P, j])[(Subscript[σ, j])] that each element is found in its specified state, such that

\(TextForm\`P\_N[σ\_1, σ\_2, …σ\_N]\  = \ ∏\+\(j = 1\)\%N P\_j[σ\_j]\)

The disorder is the summation over all possible N-body configurations

\(TextForm\`D = \(\(-\(∑\+{σ\_j}P[{σ\_j}]\ ln\ P[{σ\_j}]\)\) = \(\(-\(∑\+{σ\_j}\((\((∏\+j P\_j[σ\_j])\) \(∑\+\(k = 1\)\%N 
                      ln\ P\_k[σ\_k]\))\)\)\) = \(-\ \(∑\+{σ\_j}\(∑\+\(k = 1\)\%N\((ln\ P\_k[σ\_k] \(∏\+j P\_j[σ\_j]\)\ )\)\)\)\)\)\)\)

Each term in the product with j ≠ k can be summed over all values of (Subscript[σ, j]), reducing to a factor of unity.  There are N terms with j = k, such that the total disorder reduces to a summation over the separate contribution of each element

\(TextForm\`statistical\ independence\ \ \ ⟹\ \ \ \ D = \(∑\+\(j = 1\)\%N D\_j\ \ \ \ \ \ where\ \ \ \ \ \ \ \ D\_j = \(-\ \(∑\+\(σ\_j\)P\_j[σ\_j] 
              ln\ P\_j[σ\_j]\)\)\)\)

where the summation in (Subscript[D, j]) spans the states available to element j.  If each element has the same spectrum of states and the same probability distribution, the net disorder is simply N times the disorder for a single element.

\(TextForm\`N\ identical\ elements\ \ \ ⟹\ \ \ D = \(N\ D\_1\ \ \ \ \ where\ \ \ \ \ D\_1 = \(-\(∑\+\(σ\_1\)P\_1[σ\_1] ln\ P\_1[σ\_1]\)\)\)\)

Postulates of Quantum Statistical Mechanics

In quantum mechanics we analyze the behavior of a system by dividing the universal hamiltonian \(TraditionalForm\`\(ℋ\&^\)\) into three parts

\(TextForm\`\(ℋ\&^\)\  = \ \(H\&^\)\  + \ \(H\&^\)\^′\  + \ Δ \( H\&^\)\)

where \(TraditionalForm\`H\&^\) describes the system of interest, \(TraditionalForm\`H\&^\^′\) describes its environment (the remainder of the universe), and \(TraditionalForm\`Δ H\&^\) describes the coupling between them.  Let (Subscript[ψ, i])[x]=⟨x|(Subscript[ψ, i])⟩ represent a complete orthonormal set of wave functions spanning the Hilbert space defined by \(TraditionalForm\`H\&^\) in terms of the set of coordinates x describing the system and let (Subscript[ϕ, j])[y]=⟨y|(Subscript[ϕ, j])⟩ represent a complete orthonormal set of wave functions for the Hilbert space defined by \(TraditionalForm\`H\&^\^′\) in terms of the environment variables y.  The universal wave function

Ψ[x, y, 
        t]  =  (∑_(i, i))(Subscript[c, i, j])[t] (Subscript[ψ, i])[x] (Subscript[ϕ, j])[
            y]

can then be expanded in terms of product states (Subscript[ψ, i]) (Subscript[ϕ, j]) with coefficients

(Subscript[c, i, j])=⟨(Subscript[ψ, i]) (Subscript[ϕ, j])|Ψ⟩

Alternatively, we can focus our attention upon the system by absorbing the environment wave function into the expansion coefficients by defining

Ψ[x, y, 
        t]  =  (∑_i) (Subscript[c, i])[y, t] (Subscript[ψ, i])[x]

where

(Subscript[c, i])[y, 
        t]  =  (∑_j) (Subscript[c, i, j])[t] (Subscript[ϕ, j])[y]

hides our ignorance about the state of the universe in an implicit dependence of (Subscript[c, i]) upon y, which is treated as a random multidimensional variable.

Let the hermitian operator \(TraditionalForm\`A\&^\) acting only upon the system variables x represent an observable property of the system and consider an ensemble consisting of many identical universes prepared in the same quantum state Ψ.  The average value of the observable \(TraditionalForm\`\(A\&^\)\) obtained from instantaneous measurements performed upon each member of the ensemble (at the same time) would then be the expectation value

A[t] = ((⟨Ψ|\(A\&^\)|Ψ⟩)/(⟨Ψ|Ψ⟩)) =(( (∑_(i, j))\(\(c\_i\%*\)[y, t]\)(Subscript[c, j])[y, t]⟨(Subscript[ψ, i])|\(A\&^\)|(Subscript[ψ, j])⟩)/(\(∑\+i\( c\_i\%*\)[y, 
                  t] c\_i[y, t]\)))

However, we are less interested in the instantaneous expectation value of \(TraditionalForm\`\(A\&^\)\) than in its value averaged over a time interval that is short compared with the resolving time of the apparatus (macroscopically short) but long compared with quantum fluctuation rates (microscopically long) such that the environment variables are averaged over many states.  Thus, we define the thermodynamic variable as

\(A\&_\) = \(\(A[t]\)\&_\)  =(( (∑_(i, j))(Subscript[ρ, j, i])⟨(Subscript[ψ, i])|\(A\&^\)|(Subscript[ψ, j])⟩)/((∑_i) (Subscript[ρ, i, i])))=\(\(Tr\ ρ\&^\ A\&^\)\/\(Tr\ ρ\&^\)\)

where the statistical matrix (or density matrix) (Subscript[ρ, i, j]) is defined as

\(TextForm\`ρ\_\(i, j\)\  = \(c\_i[y, t]\ \(c\_j\%*\)[y, t]\)\&_\)

The time average is taken with respect to an interval that is macroscopically short but microscopically long.

Thus far we have avoided making any approximations or introducing any new assumptions or postulates beyond those already contained in quantum mechanics.  However, to make further progress we must introduce two postulates concerning the properties of the statistical matrix.  These postulates are most easily formulated in the energy representation, such that

\(TextForm\`	\(H\&^\)\ ψ\_i = \(E\_i\) ψ\_i\)
\(TextForm\`	\(H\&^\)\^′\ ϕ\_j = \(E\_j\%′\) ϕ\_j\)
\(TextForm\`	\(ℋ\&^\)\ Ψ\  = \ 
      E\_T\ Ψ\ \ \ with\ \ \ E\_T ≈ 
        E\_i + \ E\_j\%′\)

We assume that the coupling between the system and its environment is sufficiently weak that the energy of the system is found within a macroscopically narrow range \(TraditionalForm\`E - \(1\/2\) Δ E ≤ E\_i ≤ 
        E + \(1\/2\) Δ E\) containing many possible states of the system.  Transitions between these states are mediated by \(TraditionalForm\`Δ \(H\&^\)\).  All states within this energy range which can be connected by \(TraditionalForm\`Δ \(H\&^\)\) are considered accessible.  We assume that the environment is sufficiently complex, its states so numerous, and its transitions so rapid that phase relationships between different states of the system cannot be maintained over microscopically long time intervals.  Thus we introduce the postulate of random phases.

Furthermore, we assume that the most probable equilibrium macrostate of the system maximizes its disorder, such that all accessible microstates occur with equal probability.

We will soon find that this postulate, commonly known as the statistical postulate, is incredibly powerful.

The most important consequence of the postulate of random phases is that thermodynamic quantities can be represented by ensemble averages, such that

\(TextForm\`A\&_\  = \ \(\(Tr\ \(ρ\&^\)\ \(A\&^\)\)\/\(Tr\ \(ρ\&^\)\)\  = \ ∑\+i\( P\_i\) A\_i\)\)

is diagonal in the energy representation.  The probability (Subscript[P, i]) can now be interpreted as the frequency that microstate i occurs in the ensemble and (Subscript[A, i])=⟨(Subscript[ψ, i])|\(A\&^\)|(Subscript[ψ, i])⟩ is the quantum mechanical expectation value for the operator \(TraditionalForm\`A\&^\) in system state i.  Thus, an average with respect to a complicated but very rapid time dependence is replaced by an incoherent average over the properties of stationary states.  Having eliminated the interference between different states of the system, we can imagine that the ensemble consists of a collection of stationary states of the system, without regard to their coupling to the environment.  The postulate of random phases requires there to be some coupling, otherwise degenerate states would necessarily interfere, but once the interferences are eliminated we can pretend that each member of the ensemble is isolated and, hence, is in a stationary state.  The problem of temporal averaging then reduces to counting states and determining frequencies.  Without interference the wave functions for different members of the ensemble become disentangled and can be viewed as distinct copies of the same system.

The loss of coherence between members of an ensemble with random phases finds an analogy in the description of an ordinary beam of light in terms of a superposition of wavelets with random phases or a stream of photons with random phases.  The latter clearly comprise a statistical ensemble.  Even if the light were nearly monochromatic, with a very narrow range of frequencies, ordinary light sources are composed of many independent emitters with uncorrelated phases such that the stream of photons becomes a collection of nearly identical particles with random phases which eliminate interference phenomena.  In order to observe interference using white light one must split the beam and then recombine the subbeams using only a small difference in path length.  Similarly, in the two slit experiment, either with light or electrons, attempts to determine which slit through which a particle passes disturbs the phase relationship between the two subensembles (slits) and destroys the interference pattern.  Randomization of the phase by an external influence, here the environment the system interacts with, destroys the coherence between members of an ensemble.  Observables then become incoherent averages over the ensemble.

Evolution of Disorder

Consider a system which is very nearly isolated, interacting only weakly with its environment.  We suppose that there exists a very large number of accessible states within a narrow energy range.  Transitions between these states are mediated by the weak coupling to the external environment as represented by a hermitian interaction hamiltonian that is invariant with respect to time reversal.  The principle of detailed balance then requires the transition probability (Subscript[W, i, j]) between a pair of states to be symmetric between forward and reverse directions, such that (Subscript[W, j, i]) = (Subscript[W, i, j]).  The probability (Subscript[P, i])[t] that the system will be found in state i at time t then evolves according to

\(TextForm\`dP\_i\/dt = \ ∑\+\(j ≠ i\)\(W\_\(i, j\)\) 
            P\_j\  - \ ∑\+\(j ≠ i\)\(W\_\(j, i\)\) 
            P\_i\)

where the first term represents the transition rate into state i from all other states while the second is the rate out of state i into all other states j ≠ i.  Thus, using the principle of detailed balance, we obtain the master equation

d((Subscript[P, i])/d)t =  (∑_j) 
              (Subscript[W, i, j]) ((Subscript[P, j]) - (Subscript[P, i]))

which describes the irreversible behavior of the system.

The time dependence of D can now be analyzed using the master equation, whereby

\(TextForm\`dD\/dt = \(\(-\(∑\+i\((1 + 
                  ln\ P\_i)\) dP\_i\/dt\)\)\  = \ \(-\ \ \(∑\+\(i, j\)\(W\_\(i, j\)\) \((P\_j - P\_i)\) \((1 + 
                  ln\ P\_i)\)\)\)\)\)

Alternatively, interchanging the summation indices and using the symmetry of (Subscript[W, i, j]), we find

\(TextForm\`dD\/dt = \ \(\(-\ \ \(∑\+\(i, j\)\(W\_\(j, i\)\) \((P\_i - P\_j)\) \((1 + 
                  ln\ P\_j)\)\)\)\  = \ ∑\+\(i, j\)\(W\_\(i, 
                j\)\) \((P\_j - P\_i)\) \((1 + 
                ln\ P\_j)\)\)\)

Combining these two forms, we obtain

\(TextForm\`dD\/dt = \(1\/2\) \(∑\+\(i, j\)\(W\_\(i, j\)\) \((P\_j - P\_i)\) \((ln\ P\_j - \ 
                ln\ P\_i)\)\)\)

which is nonnegative because ln P is a monotonically increasing function of P.  Hence, we conclude that

d(D/d)t ≥ 
      0

where equality pertains if and only if (Subscript[P, i]) = (Subscript[P, j]) for all pairs of states which are connected by nonvanishing transition probabilities.  Therefore, the disorder within the system increases monotonically until an equilibrium condition is reached for which all accessible states are equally likely.  Thus, we have justified the statistical postulate of equal a priori probabilities by demonstrating that systems naturally evolve toward equilibrium conditions characterized by maximum disorder consistent with the external constraints upon them.

It may appear paradoxical that irreversible behavior is predicted for macroscopic systems whose microscopic interactions are invariant under time reversal.  Although irreversibility is a universal aspect of our experience, where order inevitably yields to chaos, how does it arise from the Schrödinger equation and the principle of detailed balance?  Note that since all quantities appearing in the master equation are real and the time dependence is linear, the master equation is not invariant under time reversal (t -> -t) or trajectory reversal (t -> -t combined with complex conjugation).  The origin of this irreversibility may be traced to the neglect of phase relationships between the system and its environment, due ultimately to our ignorance of the precise behavior of the environment (universe).  The loss of coherence between the states of the system and its environment implied by the postulate of random phases leads to the irreversibility of the master equation.

Actually, the weaker condition (∑_j)((Subscript[W, i, j]) - (Subscript[W, j, i])) = 0, which follows from causality (cf. Callen), is sufficient to ensure that (Subscript[P, i]) = (n^(-1)) in equilibrium.  Alternatively, if we were to regard the statistical postulate as fundamental, then we could deduce (Subscript[W, i, j]) = (Subscript[W, j, i]) in equilibrium, but since the transition probabilities are intrinsic properties of the dynamics of the system, we conclude that (Subscript[W, i, j]) = (Subscript[W, j, i]) for any distribution of states, be it equilibrium or not.

Finally, it is important to recognize that the equilibrium condition of equal a priori probabilities does not depend upon the transition probabilities between states.  If some state i is particularly difficult to reach because all transition probabilities (Subscript[W, i, j]) from states j ≠ i are very small, we would expect that state to be reached infrequently.  However, once it is reached, the dwell time in that state will be relatively long because all transition probabilities (Subscript[W, j, i]) which depopulate that state are also quite small.  Similarly, states which are easily reached are also quickly left.  Hence, frequency compensates dwell time so that in equilibrium the average time spent in each accessible state is equal, provided that the macroscopic averaging time is sufficiently long to ensure that all states are reached a statistically significant number of times.  Under these circumstances, the principle of detailed balance leads to equal objective probabilities as measured by temporal averages.

Statistical interpretation of thermodynamics

Entropy and energy

Thermodynamic quantities represent ensemble averages in which many possible quantum states with similar macroscopic properties contribute according to their statistical weight in the ensemble.  Recognizing that the statistical matrix evolves toward a macrostate of maximum disorder in which all microstates compatible with the external constraints appear with equal probability, the macrostate with greatest probability is that which contains the largest number of available microstates.  Let Γ represent the total number of microstates within a macrostate.  The disorder is then

\(TextForm\`D = \(\(-\(∑\+i\( P\_i\) 
              ln\ P\_i\)\)\  = \ \(\(-\ Γ\)\ ln\ Γ\^\(-1\) = \ ln\ Γ\)\)\)

where each available state is assigned equal probability (Subscript[P, i]) = (Γ^(-1)).  Finally, recognizing that the disorder function was designed to mimic all important properties of thermodynamic entropy, it is natural to make a statistical definition of entropy as

S  =  -(Subscript[k, B]) (∑_i) (Subscript[P, i]) ln (Subscript[P, i])  =  
        (Subscript[k, B]) ln Γ

where the constant of proportionality is chosen to reproduce the thermodynamic temperature scale.  Therefore, the central problem of statistical mechanics becomes the determination of the dependence of the total number of available microstates upon the external constraints.

It is instructive to express entropy

\(TextForm\`S = \(\(\(⟨\(-k\_B\) ln\ \(P\&^\)⟩\)\(\ \)\)\&_ = \ \(-k\_B\) \(∑\+i\( P\_i\) 
              ln\ P\_i\)\)\)

as an ensemble average of the quantity \(TraditionalForm\`\(-k\_B\) ln\ P\&^\) where \(TraditionalForm\`P\&^\) is an operator whose expectation value is the frequency with which a state is represented in the ensemble.  Recognizing this operator as the density operator \(TraditionalForm\`ρ\&^\), we can formally express entropy as

\(TextForm\`S = \(⟨\(-k\_B\) ln\ \(ρ\&^\)⟩\&_\  = \ \(-k\_B\) Tr\ \(ρ\&^\) 
          ln\ \(ρ\&^\)\)\)

assuming that \(TraditionalForm\`ρ\&^\) is properly normalized.  Similarly, the internal energy can be represented as an ensemble average of the energy (hamiltonian) operator, such that

\(TextForm\`U = \ \(⟨\(H\&^\)⟩\&_\ \  = \ \(∑\+i\( P\_i\) E\_i\  = \ 
          Tr\ \(ρ\&^\) \(H\&^\)\)\)\)

where (Subscript[P, i]) is the frequency that state i with energy (Subscript[E, i]) is represented in the ensemble.

Adiabatic processes

Consider a thermally isolated system upon which work may be performed by varying the external conditions without permitting heat exchange.  If we further assume that the external conditions vary sufficiently slowly so that the system is always arbitrarily close to equilibrium, transformations of state are also quasistatic.  Quasistatic adiathermal processes are described as adiabatic.  We now show that adiabatic processes do not change entropy and, hence, are reversible.

Suppose that the external conditions are changed by adjusting a parameter λ, which then becomes time dependent (λ = λ[t]).  The time dependence of entropy can then be expanded in powers of dλ/dt according to

\(TextForm\`dS\/dt = 
      A\ \((dλ\/dt)\)\^2\  + \ ⋯\)

where there is no constant term because the entropy of an equilibrium state remains constant in the absence of external changes.  Moreover, there is no linear term because d(S/d)t ≥ 
        0 independent of the sign of d(λ/d)t.  When d(λ/d)t is small, we find

\(TextForm\`dS\/dt = \(A\ \((dλ\/dt)\)\^2\ \ \ ⟹\ \ \(dS\/dλ\) = 
        A \(dλ\/dt\)\ ⟶\ 0\)\)

Thus, the entropy of a thermally insulated system is invariant with respect to adiabatic transformations.  Recognizing that ∑d(Subscript[P, i]) = 0 because the probability distribution must remain normalized to unity, entropy changes can be related to changes in microstate probability according to

\(TextForm\`dS = \(\(-k\_B\) \(∑\+i\((1 + 
                  ln\ P\_i)\) dP\_i\)\ \  = \ \(-k\_B\)\ \ \(∑\+i ln\ \(P\_i\) dP\_i\)\)\)

Hence, dS = 0 suggests that the probabilities (Subscript[P, i]) are independent of λ.  This result is a consequence of Ehrenfest's theorem, which states that adiabatic processes do not cause transitions between states, but merely adjust energies and wave functions in a smooth, parametric fashion.  Therefore, adiabatic processes preserve the distribution of microstate probabilities within an ensemble.

However, it is not always obvious when a process is sufficiently slow to be quasistatic because the equilibration or relaxation times for dissimilar systems vary considerably.  For example, adiathermal expansion or compression of a gas may be performed quite rapidly and still be adiabatic because the relaxation rate is governed by the velocity of sound.  Indeed, the conditions for adiabatic expansion of a gas are often specified as "sufficiently rapid" to neglect thermal interaction, which is often of more concern than the quasistatic requirement.  On the other hand, even with very slow variation of the magnetic field upon a ferromagnetic material, the magnetization will usually exhibit hysteresis and, hence, will not be reversible despite the extreme slowness of the variation of the external conditions.  In that case it is so difficult for a magnetic domain to spontaneously change its magnetization that the equilibration time is extremely long.

Pressure

Suppose that the hamiltonian \(TraditionalForm\`H\&^ = H\&^[λ]\) is an explicit function of the external parameter λ, where the dependence upon internal variables has been suppressed for brevity.  The thermodynamic energy U[λ] is an ensemble average of the hamiltonian and will vary with λ[t] according to

\(TextForm\`U[λ]\  = \ \(⟨H\&^[λ]⟩\&_\ \ ⟹\ \ \(dU\/dt\) = \(⟨dH\&^\/dt⟩\&_\  = \ \(⟨∂H\&^\/∂λ⟩\&_\) dλ\/dt\)\)\)

where we assume that the hamiltonian has no explicit time dependence.  On the other hand, we can consider U to be a function of entropy and λ, such that

\(TextForm\`dU\/dt = \(\((∂U\/∂λ)\)\_S\) dλ\/dt\)

where S is constant for adiabatic processes.  Comparison of these expressions now reveals that

\(TextForm\`\(\(\ \)\(\((∂U\/∂λ)\)\_S\  = \ ⟨∂\(H\&^\)\/∂λ⟩\&_\)\)\)

Thus, the quantities

\(TextForm\`\(F\&^\)\_λ\  = \ \(-\(∂\(H\&^\)\/∂λ\)\)\)

play the role of generalized forces acting upon the system and we can express variations of internal energy in the form

dU  =  
      T dS  -  (∑_λ) 
                (Subscript[F, λ]) dλ

where the thermodynamic forces are identified as ensemble averages \(TraditionalForm\`F\_λ = ⟨F\&^\_λ⟩\&_\).

If a system is confined to volume V by external forces (walls), the pressure can be interpreted as a generalized force conjugate to the volume variable.  Thus, the fundamental relation for a simple compressible system becomes

\(TextForm\`p = \(\(-\((∂U\/∂V)\)\_S\ \ ⟹\ \ \(dU\)\) = 
        T dS\  - \ p dV\)\)

Therefore, pressure is obtained from the ensemble average

\(TextForm\`p = ⟨\(-\(∂\(H\&^\)\/∂V\)\)⟩\&_\)

which can be expressed in the energy representation as

p = - (∑_i) 
                (Subscript[P, i]) ∂((Subscript[E, i])/∂)V

For example, consider an ideal nonrelativistic gas within volume V = (L^3).  Single-particle energy levels are then given by

\(TextForm\`ε\_n = \(\(n\^2\) \(\(ℏ\^2\) π\^2\)\/\(2  m L\^2\)\ \ \ where\ \ n\^2 = 
        n\_x\^2 + n\_y\^2 + n\_z\^2\)\)

Using the chain rule

\(TextForm\`∂ε\_n\/∂V = \(∂ε\_n\/∂L\) ∂L\/∂V\)

and the derivatives

\(TextForm\`∂ε\_n\/∂L = \(\(-\(2\/L\)\) ε\_n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ∂V\/∂L = \(3\/L\) V\)\)

we immediately find that

p = (2/3) (U/V)

where \(TraditionalForm\`U = N ε\&_\) is the internal energy for an ideal gas with N particles and mean single-particle energy \(TraditionalForm\`ε\&_\).  Therefore, the pressure exerted by an ideal gas upon its container is simply two-thirds of its energy density.  This result agrees with the thermodynamics of an ideal gas, as expected.  This statistical analysis reveals that the relationship between pressure and energy density is an elementary consequence of the scaling between single-particle energy and volume.  Thus, we obtain a familiar result using a dimensional argument which is somewhat simpler than kinetic theory and can easily apply similar arguments to other systems, such as the photon gas.

The adiabatic relationship between pressure and volume also follows immediately from this analysis.  The single-particle energy levels for a classical ideal gas scale with (L^(-2)) or, equivalently, with (V^(-2/3)).  Thus pressure, which depends upon the derivative of E with respect to V, scales with (V^(-5/3)).  Therefore, we find

\(TextForm\`ε ∝ L\^\(-2\) ∝ 
        V\^\(\(-2\)/3\)\ \ ⟹\ \ \ U ∝ 
        V\^\(\(-2\)/3\)\ \ ⟹\ \ \ p ∝ 
        V\^\(\(-5\)/3\)\ \ ⟹\ \ \ p V\^\(5/3\) = constant\)

for isentropic variations which do not change the probability distribution.

These derivations involve a subtlety which may have caught your attention.  If the system is prepared in an energy eigenstate, which is supposed to be stationary, how does a nontrivial probability distribution arise for which more than a single state is represented with nonvanishing probability?  To some extent the notion of a statistical ensemble evades this issue by stipulating the existence of a vast number of copies of the system which are macroscopically identical but which are prepared in microstates with frequency distribution (Subscript[P, i]).  If there are no interactions between the systems and their environments, and each system is in a stationary state, the nontrivial probability distribution also remains stationary and the ensemble average is well defined.  However, it is a tenet of statistical mechanics that temporal averages for a particular system are equivalent to ensemble averages, which then requires that each system samples all states with probability (Subscript[P, i]).  If the container could be maintained at absolute zero in a unique quantum state that does not exchange energy with the confined particles, the probability distribution for a single system prepared in a stationary state would in principle collapse about that single state.  However, because it is not possible to reach absolute zero, the container is maintained in equilibrium at some finite temperature with its own distribution of states.  Fluctuations in the state of the container couple to the state of the prisoner via interactions which can never be completely suppressed, giving rise through energy exchange to a distribution of states with finite width.

Heat and work

Consider a system with hamiltonian \(TraditionalForm\`\(H\&^\)\).  The thermodynamic energy is the ensemble average of the expectation value of this energy operator, such that

\(TextForm\`U = \(⟨\(H\&^\)⟩\  = \ ∑\+i\( P\_i\) E\_i\)\)

where (Subscript[P, i]) is the frequency that state i with energy (Subscript[E, i]) is represented in the ensemble.  If the external conditions are changed, the change in internal energy becomes

\(TextForm\`dU = ∑\+i\((\(P\_i\) dE\_i\  + \ \(E\_i\) dP\_i)\)\)

The first term, in which the energy levels are changed without altering the probability distribution, corresponds to adiathermal, isentropic work.  The second term, in which the probabilities are adjusted without changing the energy levels, must then correspond to heat transfer at constant volume.  Hence, we identify

\(TextForm\`		δ W = \(∑\+i\( 
                P\_i\) dE\_i = \(-p\) dV\)\)
\(TextForm\`		δ Q = \(∑\+i\( 
                E\_i\) dP\_i = 
        T dS\)\)

where the statistical interpretation of pressure developed in the preceding section has been used to perform the sum over states in the definition of work.  The identification of the second term with entropy then follows from the thermodynamic relationship T = (Subscript[((∂(U/∂)S)), V]).  Recognizing that

dS  =  -(Subscript[k, B])  (∑_i) 
            ln (Subscript[P, i]) d(Subscript[P, i])

we obtain the fundamental thermodynamic relation for a simple compressible system as dU = 
        T dS - p dV and interpret

\(TextForm\`			dS\  = \ \(-k\_B\)\ \(∑\+i 
            ln\ \(P\_i\) dP\_i\)\)
\(TextForm\`			p dV\  = \(-\ \(∑\+i\( 
                P\_i\) dE\_i\)\)\)

Therefore, work is associated with changes of the energies of microstates, whereas heat is associated with changes in the probabilities of microstates.

For example, consider a gas of noninteracting particles, each of which occupies some single-particle eigenstate of the box.  The probability that the entire system is found in some state of the N-body system can be expressed in terms of the probabilities for each particle that it is found in a particular single-particle eigenstate.  The diagram below illustrates, in a rather schematic fashion, the population of some of these single-particle states.  If we change the volume of the box quasistatically, the distribution of particles among eigenstates remains the same, but work is performed because the energies of the eigenstates are changed.  If we leave the volume of the box alone but shine light on the particles, some of the particles will absorb energy from the light and be promoted to higher energy levels.  Although the energy levels remain the same, the system absorbs heat because the population of levels with higher energy has been increased at the expense of the population of lower energy levels.  Thus, heat is associated with variation of the probability distribution.  If the probability of states of higher (lower) energy is increased, then heat is absorbed (expelled).  In the diagram below adiabatic compression is followed by heating at constant volume.

Multiplicity functions

To apply the statistical postulate, we must determine the number of accessible microstates consistent with the external constraints upon the system of interest.  This number is known as the multiplicity and the dependence of multiplicity upon external conditions (such as total energy, volume, and particle number) is governed by a multiplicity function.  Hence, the central problem of statistical mechanics is the determination of the multiplicity function, or avoidance thereof, by application of theorems or methods based upon statistical reasoning.

Several measures of multiplicity are available.  Suppose for the moment that we can specify the energy E, volume V, and particle number N with infinite precision for an isolated system.  For macroscopic systems there will normally be a very large number of quantum states with precisely the same values of these external variables.  Hence, we say that the macrostate specified by (E, V, N) contains many microstates and label this number Ω[E, V, N].  The function Ω[E, V, N] can be very irregular because it reflects a spectrum of discrete states with various degeneracies that depend upon microscopic details of the structure of the system.  A somewhat smoother measure of multiplicity, at least in a relative sense, is provided by

Σ[E, V, 
        N]  =  (∑_((E^′) ≤ E))Ω[
          (E^′), V, N]

which is defined as the total number of states with energies (E^′) ≤ E at or below E.  However, since it is not possible, even in principle, to define energy with infinite precision, we are more interested in the number of states within a very small interval δ E centered upon E.  [Similar arguments can be made for V and N, but those dependencies will be suppressed for brevity.]  The energy resolution δ E should be large compared with the level spacing, so that it contains many microstates, but small compared with the rate of change of thermodynamic variables or with experimental precision.  Thus, δ E should be microscopically large but macroscopically small.  Therefore, we define

Γ[
        E]  =  ∂Σ[
                  E]/∂E δ E  =  
        g[E] δ E

as the multiplicity of microstates within an energy shell \(TraditionalForm\`E ± \(1\/2\) δ E\).  Note that the dependence of  Γ upon δ E remains implicit, but should not be forgotten.  For an appropriate choice of shell width δ E, the function Γ[E] should be sufficiently smooth for practical applications.  Similarly, the density of states

g[
        E] = ∂(Σ/∂)E

should be a smooth function of energy determined by a physical, rather than strictly mathematical, differentiation process involving finite, but quite small, rather than infinitesimal differences.

These considerations apply when the level density is large and smooth.  However, at very low temperatures details of the energy spectrum may become apparent when the thermal fluctuation energy (Subscript[k, B]) T becomes comparable to the level spacing.  Probably the most important example is the phenomenon of Bose condensation.  Under those circumstances we may need to return to Ω instead of g and analyze the population of particular quantum states more explicitly.  However, because such details are not readily apparent for most of our applications, we now consider the properties of Γ when the level density is large.

Suppose that a system has f degrees of freedom and let (Subscript[Σ, i])(ε) be the total number of states for a degree of freedom i (i = 1, f) which contributes energy \(TraditionalForm\`ε\^′ ≤ ε\) to E.  For example, an ideal gas of N noninteracting structureless particles can be described by f = 3  N momentum variables.  If ε is not too small, we expect \(TraditionalForm\`Σ\_i ∼ ε/Δε\) where Δε is the average spacing between energy levels near ε.  Furthermore, we assume that the average contribution to the total energy is the same for all available degrees of freedom, so that ε ∼ E/f.  Hence, the total number of states below a specified energy is on the order of

Σ[
        E] ∼ ((((Subscript[Σ, 1])[(E/f)]))^f)

while the number within shell δ E is about

\(TextForm\`Γ[
        E] = \(∂Σ\/∂E\) δ E ∼ 
        f\ \(Σ\_1\^\(f - 
                1\)\) \(∂Σ\_1\/∂ε\) \(∂ε\/∂E\) δ E\  ∼ \ \(Σ\_1\^\(f - 
                1\)\) \(∂Σ\_1\/∂ε\) δ E\)

For very large f ∼ (10^24), (Subscript[Σ, 1])[E/f] varies slowly with E while Γ increases extremely rapidly with increasing E because f appears as an enormous exponent.  Nevertheless,

\(TextForm\`ln\ Γ[E]\  ∼ 
      f\ ln\ Σ\_1\  + 
        ln\ \((\(∂Σ\_1\/∂ε\) δ E)\) ≈ 
      f\ ln\ Σ\_1\)

is practically independent of δ E because the inequalities

(f^(-1))≪\(\(\(\(\ \)\(∂Σ\_1\)\)\/∂ε\) δ E\)  ≪ f

render the logarithmic dependence upon δ E negligible.  Therefore, we conclude that

\(TextForm\`ln\ Γ\  ∝ \ 
      f\ ln\ E\ \ \ ⟹\ \ \ Γ ∝ E\^f\)

and

ln Γ  ≈  
      ln Ω  ≈  
      ln Σ  ≈  ln g

independent of the choice of δ E.

Evidently, it matters little which multiplicity function we choose to employ for the definition of entropy (or disorder) because

S = (Subscript[k, B]) ln Γ  ≈  
        (Subscript[k, B]) ln Ω  ≈  (Subscript[k, B]) 
          ln Σ  ≈  (Subscript[k, B]) 
          ln g

up to factors of order 1 + O((f^(-1))).  This fortuitous circumstance explains in part the success of statistical methods of thermodynamic analysis.  Furthermore, it is important to recognize that entropy, defined as the logarithm of multiplicity, is an extensive function because

f ∝ 
        N  ⟹  S ∝ 
        N  ⟹  S[E, V, N]  =  
      N s[(E/N), (V/N)]

where (E/N) and (V/N) are intensive parameters.

It may at first seem surprising that ln Σ  ≈  ln g for large N, but with N appearing as an exponent the overwhelming majority of states with energy less than or equal to E are actually found very near to E.  Classically this corresponds to the fact that the volume of an N-dimensional sphere is dominated by the region very close to the surface when N is large.

Example: binary systems

Systems consisting of N particles which may each occupy only two states comprise the simplest class of statistical models but are nevertheless rich enough to illustrate many of the central principles of statistical physics.  More importantly, several systems of physical interest belong to this class.  For example, the magnetic properties of a crystal consisting of spin (1/2) atoms can be analyzed using this model.  We assume that the atoms are confined to stationary sites on a crystal lattice and neglect thermal vibrations about these sites, concentrating upon the magnetic degrees of freedom only.

Each atomic spin can point either along or against the external magnetic field.  If each spin has magnetic moment μ, the net magnetic moment of the entire system is M = ((Subscript[N, ↑]) - (Subscript[N, ↓])) μ, where (Subscript[N, ↑]) is the number of spin-up and (Subscript[N, ↓]) is the number of spin-down atoms.  In the presence of a magnetic filed B, the energy of the spin system is

\(TextForm\`U = \(\(-\(M\& ⇀ \)\)·\(\(\(B\)\(\ \)\)\& ⇀ \) = \ \(-\((\(N\_↑\) - \(N\_↓\))\)\) μ B\)\)

where spin-up refers to the alignment along the applied field.

Since each of the N states may independently occupy either of 2 states, the total number of states is (2^N).  A list of these states can be constructed from the symbolic expansion of the N-fold product of the two possibilities for each spin, namely (((↑+↓))^N).  For example, if N = 4, we find the 16 microstates listed below.

\(TextForm\`\((\(↑\)\(+↓\))\)\^4 = \(\(↑\)\(↑\)\(↑\)\(↑\)\(+\(\(↑\)\(↑\)\(↑\)\(↓\)\(+\(\(↑\)\(↑\)\(↓\)\(↑\)\(+\(\(↑\)\(↓\)\(↑\)\(↑\)\(+\(\(↓\)\(↑\)\(↑\)\(↑\)\(+\(\(↑\)\(↑\)\(↓\)\(↓\)\(+\(\(↑\)\(↓\)\(↑\)\(↓\)\(+\(\(↑\)\(↓\)\(↓\)\(↑\)\(+\(\(↓\)\(↑\)\(↑\)\(↓\)\(+\(\(↓\)\(↑\)\(↓\)\(↑\)\(+\(\(↓\)\(↓\)\(↑\)\(↑\)\(+\(\(↑\)\(↓\)\(↓\)\(↓\)\(+\(\(↓\)\(↑\)\(↓\)\(↓\)\(+\(\(↓\)\(↓\)\(↑\)\(↓\)\(+\(\(↓\)\(↓\)\(↓\)\(↑\)\(+\(\(↓\)\(↓\)\(↓\)\(↓\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)\)

There is nothing intrinsically special about the choice of symbols being up or down arrows — we could just as well have two colors or two fruits or any other pair of distinct traits.  This type of analysis can also be extended to systems with 3 or more distinct single-particle states by using a polynomial in place of the binomial (↑+↓).

Alternatively, each state of a binary system can be specified by a unique N-digit string of 1's and 0's, a binary code.  For example, if we assign spin-up a code of 1 and spin-down a code of 0, all possible states of a 4-atom system are described by the following list of (2^4) = 16 configurations.

M = +4 μ    1111
M = +2 μ    1110 1101 1011 0111
\(TextForm\`M = \ \ \ 0\ μ\ \ \ \ 1100\ 1010\ 1001\ 0110\ 0101\ 0011\)
M = -2 μ   1000 0100 0010 0001
M = -4 μ   0000

When displayed in this format, the multiplicity profile resembles a parabolic function of the net magnetic moment.  Designating the degeneracy of an energy level with (Subscript[N, ↑]) magnetic moments aligned with the magnetic field and (Subscript[N, ↓]) = (N - (Subscript[N, ↑])) moments against B as g[(Subscript[N, ↑]), N], we find

g[4, 
        4] = 1     g[3, 
            4]  =  4    g[2, 
              4]  =  6    g[1, 4]  =  4    g[0, 4]  =  
              1

summing to a total of 16 microstates.  More generally, the degeneracy of an arbitrary state is given by the binomial coefficient

g[(Subscript[N, ↑]), N] = (({{N}, {(Subscript[N, ↑])}}))  = \(\(N!\)\/\(\(N\_↑!\) \(\((N - N\_↑)\)!\)\)\  = \ \(N!\)\/\(\(\(N\_↑\) ! \) \(\(N\_↓\) ! \)\)\)

describing the number of ways in which (Subscript[N, ↑]) distinguishable objects can be selected from a total of N distinguishable boxes.

Suppose that the system of interest contains approximately 1 mole or (Subscript[N, A]) = 6.022×(10^23) independent spins, typical of macroscopic samples.  The total number of magnetic states available to such a system is then fantastically large, about (Subscript[(2^N), A]) ∼ (10^((10^(23 )))).  A common method for impressing the reader with the magnitude of such a number is to estimate the length of typescript required to represent all of its digits.  To be a little more modern (though still dated), we'll estimate the length of magnetic tape required to carry a binary representation.  A high density tape can carry 6250 bits per inch (although most formats are actually considerably less efficient).  Therefore, 6×(10^23) bits for a single state requires about (10^10) inches or about 270 light years of tape.

Similarly, the number of states whose energy lies within some specified interval (E, E + δ E) is also very large, some appreciable fraction of the number quoted above.  For all practical (macroscopic) purposes, the differences between most of the states within this interval are negligible and, in the absence of compelling contrary information, we assume that each of those states may be found with equal probability, a priori.  Of course, the probability of any individual microstate is extremely small, being the reciprocal of the total number of states within the energy interval.  If interactions between neighboring spins causes spin-flips to occur every (10^(-12)) seconds, a typical atomic time, approximately (10^35) changes of state occur each second.  Even at this rate it is impossible to sample all possible microstates  — it would still take about (10^((10^23))) seconds to sample all microstates because (10^23) - 35  =  (10^23).  Nevertheless, we easily sample enough states to perform a meaningful ensemble average because the overwhelming majority of states are macroscopically similar.  The numbers involved in the statistics of macroscopic systems simply boggle the imagination.

It is useful to define an alignment variable x as the fractional asymmetry between spin-up and spin-down populations

\(TextForm\`\(\(x\)\(=\)\(\(\(N\_↑\) - \(N\_↓\)\)\/N\)\(\ \)\)\)

such that

(Subscript[N, ↑])  =  (N/2) (1 + 
            x)
(Subscript[N, ↓])  =  (N/2) (1 - 
            x)

represent the populations of the two spin states and U = -x N μ B = -x (Subscript[U, max]) is the internal energy.  Assuming that N, (Subscript[N, ↑]), and (Subscript[N, ↓]) are all large, we can use Stirling's formula to express the multiplicity function as

ln z!  ≈  
      z ln z  -  
        z   ⟹   ln g  ≈  
      N ( ((1 + x)/2) Log[(2/(1 + x))]  +  ((1 - x)/2) 
              Log[(2/(1 - x))] )

such that the entropy becomes

S = 
      N (Subscript[k, B]) ( (((Subscript[U, max]) - U)/(2  (Subscript[U, max]))) 
              Log[((2  (Subscript[U, max]))/((Subscript[U, max]) - U))]  +  (((Subscript[U, max]) + U)/(2 
                     (Subscript[U, max]))) 
              Log[((2  (Subscript[U, max]))/((Subscript[U, max]) + U))]  )

After some straightforward algebra, we find that the internal energy takes the form

\(TextForm\`1\/T = \(\(∂S\/∂U\)\ \ ⟹\ \ U\  = \ \(-N\)\ μ\ B\ tanh[\(μ\ B\)\/\(\(k\_B\) T\)]\)\)

At high temperature the internal energy approaches zero and the spins are found with equal probability in either orientation, such that the entropy approaches its maximum possible value:

\(TextForm\`T -> \(∞\ \ \ ⟹\ \ \ U -> \(0\ \ \ ⟹\ \ \ S -> 
          N\ \(k\_B\) ln\ 2\)\)\)

The same results can also be obtained using the definition of entropy in terms of probability.  Assuming that the atoms do not interact with each other, the orientation of each spin depends only upon temperature and the external magnetic field and is independent of the orientations of neighboring spins.  Therefore, the entropy (Subscript[S, N]) = N (Subscript[S, 1]) for N statistically independent spins is simply N times the entropy for a single spin, such that

S = N (Subscript[S, 1]) = -(Subscript[k, B]) (∑_σ) (Subscript[P, 1])[σ] 
              ln (Subscript[P, 1])[σ]

where for a binary system the summation includes spin-up and spin-down states of a single atom.  Probabilities for the two-state problem can now be represented using an alignment variable x defined by

(Subscript[P, ↑])  =  ((1 + x)/2)
(Subscript[P, ↓])  =  ((1 - x)/2)

so that we again find

S =  
      N ( ((1 + x)/2) Log[(2/(1 + x))]  +  ((1 - x)/2) 
              Log[(2/(1 - x))] )

More thorough presentations of the thermodynamics of binary systems can be found in thermo2.nb and spin-half.nb.

Example: confined systems

Consider the states available to a particle in a cube of volume (L^3).  Imposing the boundary condition that the wave function vanishes at the walls, we find

ψ[x, y, z] ∝ 
      sin[(Subscript[k, x]) x] sin[(Subscript[k, y]) y] sin[(Subscript[k, z]) z]

where (Subscript[k, j]) = (((Subscript[n, j]) π)/L)for integer (Subscript[n, j]).  The single-particle energy levels are then

\(TextForm\`ε\_n = \(\(\(n\^2\) \(ℏ\^2\) π\^2\)\/\(2  m L\^2\)\ \ \ \ \ with\ \ \ \ \ \ n\^2 = 
        n\_x\^2 + n\_y\^2 + n\_z\^2\)\)

where ℏ = 1.0546×(10^(-34)) J-s is Planck's constant.  Hence, the energy levels available to a particle confined to a box are described by a discrete spectrum of the form \(TraditionalForm\`ε\_n = \(n\^2\) ε\_0\) where (Subscript[ε, 0]) = ((ℏ^2) (π^2)/2  
            m (L^2)) is the basic quantum of energy for this system.  However, many of the energy levels are shared by different states and hence are degenerate.  For example, the energy level with (n^2) = 14 carries the 6 configurations ((Subscript[n, x]), (Subscript[n, y]), (Subscript[n, z])) = (1,2,3), (3,1,2), (2,3,1), (2,1,3), (3,2,1), and (1,3,2).

Suppose that the particle is an (Subscript[N, 2]) molecule with mass m = 4.65×(10^(-26)) kg and that the box has sides of length L = 1 m.  The basic energy unit is then (Subscript[ε, 0]) = 1.18×(10^(-42))J for this system.  Further suppose that our molecule is simply one member of a mole maintained at a temperature of 273 K.  The average energy per molecule is then \(TraditionalForm\`ε\&_ = \(\(3\/2\) 
            R T/N\_A = 5.65×10\^\(-21\)\)\)J where R = 8.314J/K/mole is the molar gas constant.  Hence, ⟨(n^2)⟩ = 
        4.79×(10^21) is enormous.  Since each of the three dimensions will have the same average excitation, we estimate that the average value for each quantum number is \(TraditionalForm\`⟨n\_i⟩ ≈ 4×10\^10\).  The multiplicity for a system of (Subscript[N, A]) objects, each of whose single-particle energy levels has a large degeneracy, becomes fantastically large.  Fortunately, there are techniques which expedite the analysis of such large numbers.

Suppose that the system of interest occupies a finite volume but is not necessarily constrained to a definite volume.  We may then imagine enclosing the system in a box that is so much larger than the actual system that periodic boundary conditions can be applied with negligible effect upon the wave functions of the constituents.  From the basic principles of quantum mechanics we know that the energy levels of the system are discrete even if the spacing is infinitesimally small by macroscopic standards.  Therefore, the energy levels available to any finite system can be enumerated, at least in principle.

Example: N distinguishable oscillators

Consider a system of N harmonic oscillators and assume that interactions between these oscillators can be neglected.  For example, electromagnetic radiation in a cavity or the vibrations of atoms about lattice sites in a crystal may be analyzed in terms of independent modes of oscillation.  Each individual oscillator has an energy spectrum (Subscript[ε, n]) = 
        n ℏ ω consisting of an infinite sequence of equally spaced levels where n is the total number of quanta and ℏ ω is the fundamental quantum of energy for the system.  [The zero-point energy represents a trivial offset of the energy scale and can be omitted.]

For example, consider a single particle in a 3-dimensional harmonic well.  The total number of quanta is then n = (Subscript[n, x]) + (Subscript[n, y]) + (Subscript[n, z]) where each of the three spatial dimensions can be considered to be an independent oscillator.  The degeneracy of each single-particle energy level is equal to the total number of ways that n quanta can be distributed among the three independent axes.  There are n + 1 possible values of (Subscript[n, x]) between 0 and n.  For each of these values, we can choose (Subscript[n, y]) to be anywhere between 0 and n - (Subscript[n, x]).  The number of quanta along the z-axis is then determined.  Hence, the degeneracy of a single-particle level with n quanta is given by the sum

\(TextForm\`g[
        n] = \(∑\+\(n\_x = 0\)\%n\((n - n\_x + 
              1)\)\  = \ \(∑\+\(k = 1\)\%\(n + 1\)k\  = \ \(1\/2\) \((n + 1)\) \((n + 2)\)\)\)\)

whose value can be demonstrated by induction.

The degeneracy for a system of many independent oscillators can now be obtained by extending this argument to f independent degrees of freedom, where f is the total number of oscillator modes rather than simply the total number of particles.  Let n represent the total number of quanta, so that the internal energy of the system is U = 
        n ℏ ω.  The degeneracy is simply the number of distinct ways that n indistinguishable objects (quanta) can be distributed among f distinguishable boxes (vibrational modes).  Suppose that the boxes are represented by vertical lines marking their boundaries and that the objects are represented by circles, as sketched below for a particular configuration.

•• | •••• || ••• | ••••• || •

The outermost walls need not be included.  The number of combinations of n objects and f - 1 interior partitions is simply (f + n - 1)!.  However, because the n! permutations of the balls or the (f - 1)! permutations of the partitions among themselves leave the system invariant, the degeneracy is simply the binomial coefficient

g[n, f] =(({{f + n - 1}, {n}})) = (((f + n - 1)!)/(n! (f - 1)!))

The earlier result for a single three-dimensional oscillator is recovered by choosing f -> 3.

In the limit of large n and large f, we can employ Stirling's approximation to obtain

ln g  ≈  (f + n) ln[f + n]  -  
          n ln n  =  
      f ( (1 + x) ln (1 + x)  -  
            x ln x )

where x = n/f is the average excitation per oscillator.  The entropy can now be expressed as

S  =  (Subscript[k, B]) 
        f ( (1 + x) ln (1 + x)  -  
            x ln x )

where x = 
        U/f ℏ ω.  Evaluating the temperature, we find

\(TextForm\`1\/\(\(k\_B\) T\) = \(\(\(∂\ 
                    ln\ g\)\/∂U\)\ \ \ ⟹\ \ \ \ f\ y\  = \ \(\(∂\ ln\ g\)\/∂x\  = \ \(f\ ln\ \(\(1 + 
                      x\)\/x\)\ \ \ \ ⟹\ \ \ \ x\  = \ 
            1\/\(e\^y - 1\)\)\)\)\)

where y = ((ℏ ω)/((Subscript[k, B]) T)) is a dimensionless variable that represents the ratio between oscillator and thermal energies.  Therefore, we obtain the thermal equation of state

\(TextForm\`U\  = \ \ \(f\ ℏ\ ω\)\/\(e\^\(ℏω/k\_B  T\) - 1\)\)

and heat capacity

\(TextForm\`C\_V\/\(f\ k\_B\) = \ \(\(\(y\^2\) e\^y\)\/\((e\^y - 1)\)\^2\ \ \ \ where\ \ \ \ y = \(ℏ\ ω\)\/\(\(k\_B\) T\)\)\)

It is useful to define a dimensionless reduced temperature as τ = (y^(-1)) = (Subscript[k, B]) 
            T/ℏ ω.

Einstein's model of the heat capacity of a crystal containing N atoms in a lattice treated the 3  N normal modes of vibration as independent harmonic oscillators with a single average frequency ω.  If we define a characteristic temperature (Subscript[T, E]) = ℏ ω/(Subscript[k, B]), the reduced temperature becomes τ = T/(Subscript[T, E]).  The figure below displays the dependence of the heat capacity upon reduced temperature.  At high temperatures, τ ≫ 1, one finds the classical heat capacity (Subscript[C, V]) -> 3 N (Subscript[k, B]) expected for equipartition among 3  N coordinates and 3 N momenta.  However, at low temperatures, τ < 1, the heat capacity is much smaller quantum mechanically than classically because the finite energy required to excite a quantized oscillator freezes out states that would participate in a classical theory permitting arbitrarily small excitations.  Although this model is simplistic, the explanation of reduced heat capacity at low temperature in terms of quantization of energy provided important support for the early quantum theory.

We will return to this and more refined models shortly.  Further information on these systems can be found in the notebooks hotherm.nb, debye.nb, and planck.nb.

Thermal interaction

Consider two isolated systems with multiplicity functions (Subscript[Γ, 1])[(Subscript[E, 1])] and (Subscript[Γ, 2])[(Subscript[E, 2])].  These functions depend upon details of internal structure and may be quite different for dissimilar systems, but we assume that each (Subscript[Γ, i])[(Subscript[E, i])] is a rapidly increasing function of energy.  The composite multiplicity function for the combined system comprising the two noninteracting subsystems is then

(Subscript[Γ, 1, 2]^((0)))[(Subscript[E, 1]), 
        (Subscript[E, 2])]  =  (Subscript[Γ, 1])[(Subscript[E, 1])] (Subscript[Γ, 2])[
          (Subscript[E, 2])]

Now suppose that these systems are brought into thermal contact with each other but that the combined system remains isolated.  If the initial energies are (Subscript[E, 1  i]) and (Subscript[E, 2  i]), the initial multiplicity is \(TraditionalForm\`Γ\_\(1, 2\)\%\((i)\) = Γ\_1[E\_\(1  i\)] Γ\_2[E\_\(2  i\)]\).  As the two systems interact with each and exchange energy, the number of available microstates will also change as each system's share of the total energy changes.  Although these systems can exchange energy through their shared boundaries, we assume that there are no long-range macroscopic interactions between them so that the net energy E for the combined system is very near (Subscript[E, 1  i]) + (Subscript[E, 2  i]); hence, we assume that E = (Subscript[E, 1]) + (Subscript[E, 2]) is simply additive.  During the thermal interaction, the macrostates for each subsystem change but are confined by the band (Subscript[E, 1]) + (Subscript[E, 2]) = 
        E ± δ E where δ E is the energy resolution.  (We left δ E implicit in  (Subscript[Γ, i]).)  The total multiplicity is then

Γ[
        E]  =  (∑_k) (Subscript[Γ, 1])[(Subscript[E, 1  k])] (Subscript[Γ, 2])[
              E - (Subscript[E, 1  k])]  ≈  ∫(Subscript[g, 1])[
              (Subscript[E, 1])] (Subscript[Γ, 2])[
              E - (Subscript[E, 1])] d(Subscript[E, 1])

where the summation includes all possible energy divisions between two subsystems.  Since each term is positive and the sum includes (Subscript[Γ, 1, 2]^((i))), clearly

\(TextForm\`Γ\  ≥ \ Γ\_\(1, 2\)\%\((i)\)\)

where equality applies only in the rare circumstance that only a single macrostate (k = i) is available.  Thus, thermal interaction must increase entropy as more microstates become available with the same total energy.  Furthermore, because (Subscript[Γ, 1])[(Subscript[E, 1])] increases very rapidly as (Subscript[E, 1]) increases while (Subscript[Γ, 2])[E - (Subscript[E, 1])] decreases very rapidly as its share of the fixed total energy decreases, the product (Subscript[Γ, 1])[(Subscript[E, 1])] (Subscript[Γ, 2])[
          E - (Subscript[E, 1])] is a very sharply peaked function of (Subscript[E, 1]).  Therefore, in equilibrium the combined system will be found with overwhelming probability very near the peak of (Subscript[Γ, 1, 2])[(Subscript[E, 1]), E - (Subscript[E, 1])] characterized by most probable energies \(TraditionalForm\`E\&~\_1\) and \(TraditionalForm\`E\&~\_2\), with \(TraditionalForm\`E\&~\_1 + E\&~\_2 = E\_1 + E\_2\).  Since (Subscript[Γ, 1, 2]) is sharply peaked, the summation is dominated by the single greatest contribution \(TraditionalForm\`Γ\_1[E\&~\_1] Γ\_2[
          E - E\&~\_1]\).  Hence, maximizing ln (Subscript[Γ, 1, 2]) with respect to (Subscript[E, 1]) is equivalent to maximizing ln Γ, whereby

\(TextForm\`\(∂\/∂E\_1\) 
        ln\ Γ\_\(1, 2\)[E\_1, 
          E - E\_1]\  = \ \(0\ \ \ ⟹\ \ \(∂\/∂E\_1\) 
          ln\ Γ\_1[E\_1] = \(∂\/∂E\_2\) 
          ln\ Γ\_2[E\_2]\)\)

This relationship is illustrated in the diagram below, which displays schematic multiplicity functions (Subscript[Γ, 1])[(Subscript[E, 1])], (Subscript[Γ, 2])[E - (Subscript[E, 1])], and (Subscript[Γ, 1, 2])[(Subscript[E, 1]), E - (Subscript[E, 1])].  Note that to display all three functions together, we must multiply (Subscript[Γ, 1]) and (Subscript[Γ, 2]) by small factors (Subscript[γ, i]) and must choose the energy range and axis scaling carefully because the multiplicity functions for macroscopic functions are incredibly steep.  The product of a rapidly falling and a rapidly increasing function is usually a sharply peaked function for which the position and width of the peak depends upon the two slopes.  Another useful way to visualize the optimization of energy sharing is to display the product multiplicity function within the band permitted by the energy constraint as a density plot in which the shading indicates the number of available states.  The initial state will usually not be near the peak of the composite multiplicity function, but once the systems are permitted to exchange energy the combined system will move toward the most probable configuration and will soon be found with overwhelming probability within the peak of maximum entropy.

Recognizing that thermodynamic equilibrium requires (Subscript[T, 1]) = (Subscript[T, 2]) and using the thermodynamic relationship (1/T) = ∂(S/∂)U, we are led to the identification of

S = (Subscript[k, B]) ln Γ

as the entropy.  Boltzmann's constant (Subscript[k, B]) = 
        1.38066×(10^(-23)) J (kelvin^(-1)) is chosen to produce agreement with the thermodynamic temperature scale.  Thus, at equilibrium we find

\(TextForm\`Γ[E] ≈ \ 
        g\_1[E\&~\_1] Γ\_2[
            E - E\&~\_1] δ E\_1\ \ ⟹\ \ S = S\_1 + S\_2\)

where δ (Subscript[E, 1]) is the width of the sharp peak in (Subscript[Γ, 1, 2])[(Subscript[E, 1]), E - (Subscript[E, 1])]; clearly ln δ (Subscript[E, 1]) is negligible for macroscopic systems.  Therefore, entropy is an extensive variable, as required by thermodynamics, and

S ≥ 
      (Subscript[S, 1  i]) + (Subscript[S, 2  i])

where (Subscript[S, 1  i]) = (Subscript[k, B]) 
          ln (Subscript[Γ, 1  i]) and (Subscript[S, 2  i]) = (Subscript[k, B]) ln (Subscript[Γ, 2  i]) are the entropies of the two subsystems before interaction.  Thus, we have provided a statistical interpretation of the law of increasing entropy.

The change in entropy that results from an infinitesimal exchange of heat \(TraditionalForm\`δ Q = dE\_1\) between two systems in thermal contact via immovable and impenetrable walls which do not permit exchange of volume (work) or particles is

\(TextForm\`dS = \(dS\_1 + dS\_2 = \(k\_B\) \(\(∂ln\ Γ\_1\)\/∂E\_1\) dE\_1\  + \ \(k\_B\) \(\(∂ln\ Γ\_2\)\/∂E\_2\) \(\(d\)\(E\_2\)\(\ \)\)\)\)

such that

dS  =  ((1/(Subscript[T, 1])) - 
            (1/(Subscript[T, 2]))) δ  Q

The statistical requirement of increasing disorder as the system evolves toward a macrostate with greater multiplicity of microstates is equivalent to the thermodynamic inequalities

dS ≥ 
      0  ⟹ δ Q ≥ 
      0 when (Subscript[T, 1]) ≤ 
      (Subscript[T, 2])    or   δ Q ≤ 
      0 when (Subscript[T, 1]) ≥ (Subscript[T, 2])

which require heat to flow spontaneously from warmer to colder systems.  Furthermore, we identify the thermodynamic internal energies (Subscript[U, i]) with the equilibrium energies \(TraditionalForm\`E\&~\_i\) which maximize the net entropy.

More generally, suppose that two interacting systems can exchange energy, volume, and particles through a flexible, permeable, diathermal boundary.  Once again assume that Γ is sharply peaked about \(TraditionalForm\`Γ\_1[E\&~\_1, \(V\_1\)\&~, 
          N\&~\_1] Γ\_2[E\&~\_2, \(V\_2\)\&~, N\&~\_2]\) where \(TraditionalForm\`E\&~\_i\), \(TraditionalForm\`V\&~\_i\), and \(TraditionalForm\`N\&~\_i\) are the most probable values for the energy, volume, and particle number for each subsystem subject to the constraints \(TraditionalForm\`E\&~\_1 + E\&~\_2 = 
        E ± δ E\),  \(TraditionalForm\`V\&~\_1 + V\&~\_2 = 
        V ± δ V\), and  \(TraditionalForm\`N\&~\_1 + N\&~\_2 = 
        N ± δ N\).  Thus, maximization of ln Γ with respect to independent variations of (Subscript[E, 1]), (Subscript[V, 1]), and (Subscript[N, 1]) is equivalent to maximizing ln (Subscript[Γ, 1, 2]) and results in the equilibrium conditions

\(TextForm\`\(∂ln\ Γ\_1\)\/∂E\_1 = \(∂ln\ Γ\_2\)\/∂E\_2\)
\(TextForm\`\(∂ln\ Γ\_1\)\/∂V\_1 = \(∂ln\ Γ\_2\)\/∂V\_2\)
\(TextForm\`\(∂ln\ Γ\_1\)\/∂N\_1 = \(∂ln\ Γ\_2\)\/∂N\_2\)

specifying the most probable macrostate.  Correspondence with the fundamental relation of thermodynamics, T dS = dU  +  
          p dV  -  μ dN is then achieved by means of the identifications

\(TextForm\`		1\/T = \(\((∂S\/∂U)\)\_\(V, N\)\ \ \ \ \ \ \ \ \ ⟺\ \ β\  = \ \((\(∂ln\ Γ\)\/∂E)\)\_\(V, N\)\)\)
\(TextForm\`		p\/T = \(\((∂S\/∂V)\)\_\(U, N\)\ \ \ \ \ \ \ \ \ ⟺\ \ β\ p\  = \ \((\(∂ln\ Γ\)\/∂V)\)\_\(E, N\)\)\)
\(TextForm\`		μ\/T = \(\(-\((∂S\/∂U)\)\_\(V, N\)\ \ \ \ \ ⟺\ \ β\)\ μ = \(-\ \((\(∂ln\ Γ\)\/∂N)\)\_\(E, V\)\)\)\)

where S = (Subscript[k, B]) ln Γ is entropy, T temperature, p pressure, and μ chemical potential and where it is convenient to define β = ((((Subscript[k, B]) T))^(-1)).  Equilibrium then requires

\(TextForm\`T\_1 = \(T\_2\ \ \ \ \ \ \ \ p\_1 = \(p\_2\ \ \ \ \ \ \ \ μ\_1 = μ\_2\)\)\)

for the final, most probable, macrostate with \(TraditionalForm\`U\_i ≈ E\&~\_i\), \(TraditionalForm\`V\_i ≈ V\&~\_i\), and \(TraditionalForm\`N\_i ≈ N\&~\_i\).  Therefore, an intensive parameter determines the equilibrium distribution of a conserved quantity between interacting subsystems.  Note that the equalities between the extensive variables (U, V, N) and their most probable values are to be interpreted in a thermodynamic sense, which requires these quantities to be found with overwhelming probability within very narrow ranges centered upon their most probable (equilibrium) values.

Fluctuations

The widths of the distributions for the thermodynamic variables can be deduced from a Gaussian approximation to Γ.  For simplicity, consider only the dependence upon energy.  Suppose that two systems are in thermal contact with each other but that the composite system is thermally isolated.  The two systems share the net total energy, such that E = (Subscript[E, 1]) + (Subscript[E, 2]).  Near equilibrium we can expand the logarithm of the composite multiplicity function about its peak, for which the subsystem energies are near their most probable values, \(TraditionalForm\`E\&~\_1\) and \(TraditionalForm\`E\&~\_2\) with \(TraditionalForm\`E\&~\_1 + E\&~\_2 = E\).  Hence, we write

\(TextForm\`ln\ Γ\_\(1, 2\)[E\_1]\  ≈ \ 
      ln\ Γ\_\(1, 2\)[
            E\&~\_1]\  + \ \(1\/2\) \(\((\(∂\^2 ln\ Γ\_\(1, 2\)\)\/∂E\_1\^2)\)\_\(E\_1 = 
                E\&~\_1\)\) \((E\_1 - E\&~\_1)\)\^2 + \ ⋯\)

where \(TraditionalForm\`Γ\_\(1, 2\)[
          E\&~\_1] = Γ\_1[E\&~\_1] Γ\_2[
            E - E\&~\_1]\), such that

\(TextForm\`Γ\_\(1, 2\)[
        E\_1] ≈ Γ\_\(1, 2\)[
          E\&~\_1]\ Exp[\(-\(\((E\_1 - E\&~\_1)\)\^2\/\(2  σ\^2\)\)\)]\)

We can now identify

\(TextForm\`σ\^\(-2\)\  = \ \(\(-\ \((\(∂\^2 ln\ Γ\_\(1, 2\)\)\/∂E\_1\^2)\)\_\(E\_1 = E\&~\_1\)\)\  = \(-\ \((\(∂\^2 ln\ Γ\_1\)\/∂E\_1\^2\  + \(∂\^2 ln\ Γ\_2\)\/∂E\_2\^2\ )\)\_\(E\_1 = E\&~\_1, E\_2 = E\&~\_2\)\)\)\)

as the energy variance for the composite system.  It is useful to identify the individual contributions as

\(TextForm\`σ\_i\^\(-2\)\  = \ \(\(-\ \(\(∂\^2 
                  ln\ Γ\_i\)\/∂U\_i\^2\)\)\  = \ \(\(-\(1\/k\_B\)\) ∂\^2 S\_i\/∂U\_i\^2\  = \ \((\(k\_B\) \(T\^2\) C\_i)\)\^\(-1\)\)\)\)

where (Subscript[C, i]) is the appropriate (here, isochoric) heat capacity for system i.  Thus, the mean energy fluctuations for two interacting systems are related to those for each individual system by

(1/(σ^2)) = 
      (1/((Subscript[σ, 1])^2))  +  (1/((Subscript[σ, 2])^2))

such that

(σ^2) = (Subscript[k, B]) 
        (T^2) (((Subscript[C, 1]) (Subscript[C, 2]))/((Subscript[C, 1]) + (Subscript[C, 2])))

If one system is very much smaller than the other, the smaller system dominates the energy width, such that

\(TextForm\`C\_2 ≫ 
      C\_1\ \ ⟹\ \ σ\^2 ≈ \ \(k\_B\) \(T\^2\) \(C\_1\) \((1 - C\_1\/C\_2)\)\)

Hence, the average energy is very near the most probable energy, and our other approximations are valid also, if

(σ/U)≪1  ⟹  (Subscript[k, B])  (T^2) 
          (Subscript[C, V])≫ (U^2)

where (Subscript[C, V]) is associated with the smaller subsystem.  For example, if our system is a classical ideal gas, we find

\(TextForm\`U = \(\(3\/2\) N \(k\_B\) 
          T\ \ ⟹\ \ σ = \(\(\((\(3\/2\) 
                    N)\)\^\(1/2\)\) \(k\_B\) 
            T\ \ ⟹\ \(σ\/U\) = \((\(3\/2\) N)\)\^\(\(-1\)/2\)\)\)\)

such that (σ/U) ≪ 1for large N.  Thus, if N ∼ (10^24), the relative width of the energy distribution is about (10^(-12)) and it is extremely unlikely that the system can be found with an energy differing from its most probable value by more that one part in (10^11).  Therefore, the energy distribution is extremely sharp in a macroscropic sense and the thermodynamic energy is extremely close to the most probable energy despite the rapid changes in the microstate of the system because there are a vast number of states within an extremely narrow band about the most probable energy.

For each subsystem, the root-mean-square (rms) energy fluctuation

(Subscript[σ, i])[
        (Subscript[U, i])]  =  ((((Subscript[k, B]) (T^2) (Subscript[C, i])))^(1/2))

is proportional to temperature and to the square-root of its heat capacity.  Recognizing that the heat capacity is proportional to the size of a system, it is useful to express the heat capacities as (Subscript[C, i]) = (Subscript[m, i]) (Subscript[c, i]) where (Subscript[m, i]) is the mass and (Subscript[c, i]) the specific heat for system i, such that

\(TextForm\`σ\_i[
        U\_i]\  = \ \(\((\(k\_B\) \(T\^2\) \(m\_i\) c\_i)\)\^\(1/2\)\ \ ⟹\ \ \ σ\^2 = \(k\_B\) 
          T\^2\ \(\(m\_1\) \(c\_1\) \(m\_2\) c\_2\)\/\(\(m\_1\) c\_1 + \(m\_2\) c\_2\)\)\)

Suppose that the two systems have similar composition but different sizes, such that

(Subscript[c, 1]) ≈ 
        (Subscript[c, 2])  ⟹  (σ^2) = (Subscript[k, B]) 
        (T^2) (((Subscript[m, 1]) (Subscript[m, 2]))/((Subscript[m, 1]) + (Subscript[m, 2])))

Similar analyses can also be performed for fluctuations of volume, particle number, or other variables.  For example, one finds that fluctuations of volume are governed by the compressibility of the system.

This argument can be generalized to include any extensive quantity that can be exchanged between a system and a much larger reservoir.  Variations of the total entropy for two interacting systems can be expressed in the form

\(TextForm\`dS\  = \ \(dS\_1\  + \ dS\_2\  = \ dS\_1 + \ \(∂S\_2\/∂E\_2\) dE\_2\  + \ \(∂S\_2\/∂V\_2\) dV\_2\  + \ ⋯\)\)

where the variation in the contribution d(Subscript[S, 2]) of the reservoir is expanded in terms of its extensive variables; here we limit the expansion to two terms, for simplicity, but others can be included easily.  Using

\(TextForm\`E = \(E\_1 + 
          E\_2\ \ \ ⟹\ \ \ \(dE\_2\) = \(-dE\_1\)\)\)
\(TextForm\`V = \(V\_1 + 
          V\_2\ \ \ ⟹\ \ \(dV\_2\) = \(-dV\_1\)\)\)

for the conserved extensive quantities and identifying the intensive variables

\(TextForm\`∂S\_2\/∂E\_2 = \(1\/T\_2\ \ \ \ \ \ \ ∂S\_2\/∂V\_2\  = \ 
        p\_2\/T\_2\)\)

we find

\(TextForm\`dS\  = \ dS\_1\  - \ \(dE\_1 + \(p\_2\) dV\_1\)\/T\_2\)

Assume that system 2 is a large reservoir and the system 1 is a much smaller subsystem.  The temperature and pressure of the reservoir are constant for all practical purposes, unaffected by changes is the energy or volume of the much smaller subsystem.  Thus, the entropy change for a small but finite fluctuation of the extensive variables for the subsystem takes the form

\(TextForm\`Δ S\  = \ \(T\_2 Δ S\_1\  - Δ E\_1\  - p\_2 Δ V\_1\)\/T\_2\)

The probability for such a fluctuation is then

\(TextForm\`P\  ∝ \ 
      Exp[\(T\_2 Δ S\_1\  - Δ E\_1\  - p\_2 Δ V\_1\)\/\(\(k\_B\) T\_2\)]\)

Focusing our attention upon the smaller subsystem, it is convenient to identify T = (Subscript[T, 2]) and p = (Subscript[p, 2]) and to omit the subscript for the subsystem, such that

\(TextForm\`P\  ∝ \ 
      Exp[\(T Δ S\  - Δ E\  - p Δ V\)\/\(\(k\_B\) T\)]\)

Next expand the variation of energy about its most probable value, such that

\(TextForm\`Δ E\  ≈ \ \(∂E\/∂S\) Δ S\  + \ \(∂E\/∂V\) Δ V\  + \ \(1\/2\) \((\ \(∂\^2 
                      E\/∂S\^2\) \((Δ S)\)\^2\  + 
              2\ \ \((∂\^2 E\/\(∂V ∂S\))\) Δ V\ Δ S\  + \ \ \(∂\^2 
                      E\/∂V\^2\) \((Δ V)\)\^2)\)\  + \ ⋯\)

and identify

\(TextForm\`∂E\/∂S\  = \ \(T\ \ \ \ \ \ \ \ ∂E\/∂V = \(-\(p\/T\)\)\)\)

to obtain a Gaussian probability distribution

\(TextForm\`P\  ∝ \ 
      Exp[\(-β\) \(1\/2\) \((\ \(∂\^2 
                      E\/∂S\^2\) \((Δ S)\)\^2\  + 
              2\ \ \((∂\^2 E\/\(∂V ∂S\))\) Δ V\ Δ S\  + \ \ \(∂\^2 
                      E\/∂V\^2\) \((Δ V)\)\^2)\)\ ]\)

governing entropy and volume fluctuations in the subsystem.  Notice that the first-order variations are eliminated by the equilibrium conditions that require the temperature and pressure of the most probable state of the subsystem to be equal to those of the reservoir.  Using

\(TextForm\`Δ T\ Δ S\  = \ \((\(∂T\/∂S\) Δ S\  + \ \(∂T\/∂V\) Δ V)\) Δ S\)
\(TextForm\`Δ p\ Δ V\  = \ \((\(∂p\/∂S\) Δ S\  + \ \(∂p\/∂V\) Δ V)\) Δ V\)

and the Maxwell relation

\(TextForm\`\((∂p\/∂S)\)\_V\  = \ \(-\((∂T\/∂V)\)\_S\)\)

we observe

\(TextForm\`Δ T\ Δ S\  - \ Δ p\ Δ V\  = \ \(∂\^2 
                E\/∂S\^2\) \((Δ S)\)\^2\  + 
        2\ \ \((∂\^2 E\/\(∂V ∂S\))\) Δ V\ Δ S\  + \ \ \(\(∂\^2 
                E\/∂V\^2\) \(\((Δ V)\)\^2\)\(\ \)\)\)

and express the probability distribution for fluctuations in the form

\(TextForm\`P\  ∝ \ 
      Exp[\(-\(\(Δ T\ Δ S\  - \ Δ p\ Δ V\)\/\(2 \( k\_B\) T\)\)\)]\)

where two of the variables are considered independent and the other two dependent.

In the absence of an entropy meter, it is useful to expand the entropy fluctuation

\(TextForm\`Δ S\  = \ \(\(\((∂S\/∂T)\)\_V\) Δ T\  + \ \(\((∂S\/∂V)\)\_T\) Δ V\  = \ \(C\_V\/T\) Δ T\  + \ \(\((∂p\/∂T)\)\_V\) Δ V\)\)

in terms of temperature and volume and use the the equation of state for pressure fluctuations

\(TextForm\`Δ p\  = \ \(\(\((∂p\/∂T)\)\_V\) Δ T\  + \ \(\((∂p\/∂V)\)\_T\) Δ V\  = \ \(\((∂p\/∂T)\)\_V\) Δ T\  - \ \(Δ V\)\/\(V\ κ\_T\)\)\)

such that

\(TextForm\`Δ T\ Δ S\  - \ Δ p\ Δ V\  = \ \(C\_V\/T\) \((Δ T)\)\^2\  + \ \((Δ V)\)\^2\/\(V\ κ\_T\)\)

The probability distribution for fluctuations

\(TextForm\`P\  ∝ \ 
      Exp[\ \(-\(C\_V\/\(\(k\_B\) 
                    T\^2\)\)\) \((Δ T)\)\^2\/2] 
        Exp[\(-\(1\/\(\(k\_B\) 
                    T\ V\ κ\_T\)\)\) \((Δ V)\)\^2\/2]\)

factors into two Gaussians.  Therefore, we conclude that temperature and volume fluctuations are statistically independent and can identify the variances for these fluctuations

\(TextForm\`⟨\((Δ T)\)\^2⟩\  = \ \(\(\(k\_B\) T\^2\)\/C\_V\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ⟨\((Δ V)\)\^2⟩\  = \ \(\(k\_B\) 
            T\ V\ κ\_T\ \ \ \ \ \ \ \ \ \ \ \ \ \ ⟨Δ T\ Δ V⟩\  = \ 0\)\)\)

by inspection.  When analyzing local fluctuations of density, it is more natural to employ a subsystem with fixed volume and variable particle number that fixed particle number and variable volume — the local subsystem is fixed in space with imaginary walls that permit free passage of particles in or out as the density changes.  The fluctuation in density, ϱ = N/V, is obtained using

\(TextForm\`ϱ\  = \ \(\(N\/V\)\ \ ⟹\ \ \ \ Δ ϱ\  = \ \(\(-\(ϱ\/V\)\) Δ V\ \ \ \ ⟹\ \ ⟨\((Δ ϱ)\)\^2⟩\  = \ \(ϱ\^2\/V\^2\) ⟨\((Δ V)\)\^2⟩\)\)\)

such that

\(TextForm\`⟨\((Δ ϱ)\)\^2⟩\  = \ \(ϱ\^2\/V\) \(k\_B\) 
        T\ κ\_T\)

is proportional to the isothermal compressibility.

For a simple ideal gas we obtain

\(TextForm\`ideal\ gas\ \ ⟹\ \ ⟨\((Δ T)\)\^2⟩\  = \ \(2  T\^2\)\/\(3  N\)\ , \ \ \ \ \ \ \ \ ⟨\((Δ V)\)\^2⟩\  = \ 
      V\^2\/N\ , \ \ \ \ \ ⟨\((Δ ϱ)\)\^2⟩\  = \ ϱ\^2\/N\)

Thus, for an intensive variable, like T or ϱ, the variance is inversely proportional to N while for an extensive variable, like V, the variance is directly proportional to N.  Nevertheless, the relative fluctuations  in both extensive and intensive variables

\(TextForm\`ideal\ gas\ \ ⟹\ \ \(⟨\((Δ T)\)\^2⟩\^\(1/2\)\/T\)\  = \ \@\(2\/3\)\ \ N\^\(\(-1\)/2\)\ , \ \ \ \ \ \ \ \ ⟨\((Δ V)\)\^2⟩\^\(1/2\)\/V\  = \ 
      N\^\(\(-1\)/2\)\ , \ \ \ ⟨\((Δ ϱ)\)\^2⟩\^\(1/2\)\/ϱ\  = \ N\^\(\(-1\)/2\)\)

scale with (N^(-1/2)) and are usually very small for macroscopic subsystems except near a phase transition where one or more of the relevant response functions might diverge.

Problems

⟨((Overscript[k, ⇀])^′)|\(H\&^\)|(Overscript[k, ⇀])⟩ = ε[k] δ[(Overscript[k, ⇀]), 
                ((Overscript[k, ⇀])^′)]

ε[
        k]  =  (((ℏ^2) (k^2))/(2  m))

⟨(Overscript[r, ⇀])|(Overscript[k, ⇀])⟩ = \(\(V\^\(\(-1\)/
                    2\)\) e\^\(i\ k\&⇀·r\&⇀\)\)

(Overscript[k, ⇀])  =  {(Subscript[n, x]), (Subscript[n, y]), 
          (Subscript[n, z])} ((2  π)/L)

\(TextForm\`∑\+\(n\_x, n\_y, n\_z\)\ \ \ \ \(\(⟶\)\(\ \ \ \ \ \)\(V\)\) \(∫d\^3 k\/\((2  π)\)\^3\)\)

ln Γ  ≈  
      ln Ω  ≈  
      ln Σ  ≈  ln g

(Subscript[V, N])[
        R]  =  ∫d(Subscript[V, N]) Θ[ 
            (R^2) - (∑_(i = 1)^N) ((Subscript[x, i])^2) ]

\(TextForm\`dV\_N\  = \ ∏\+\(i = 1\)\%Ndx\_i\)

\(TextForm\`∫\_\(-∞\)\%∞\( e\^\(-x\^2\)\) dx\  = \ \(π\^\(1/2\)\ \ ⟹\ \ \(∫\_\(-∞\)\%∞\( e\^\(-r\^2\)\) dV\_N\)\  = \ π\^\(N/2\)\)\)

((((1/(1 - t))))^f)  =  (∑_n) g[n, f] 
          (t^n)

T > 
      0  ⟹  ∂(S/∂)E  >  
      0

\(TextForm\`C\_V ≥ 
      0\ \ ⟹\ \ \((∂E\/∂T)\)\_V ≥ 0\)

Ω[E, V, N] ~ 
          Exp[ N ϕ[(E/N), (V/N)] ]  ⟹  S~
          N (Subscript[k, B]) ϕ[(E/N), (V/N)]

\(TextForm\`⟨\((Δ T)\)\^2⟩\  = \ \(\(\(k\_B\) T\^2\)\/C\_V\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ⟨\((Δ V)\)\^2⟩\  = \ \(\(k\_B\) 
            T\ V\ κ\_T\ \ \ \ \ \ \ \ \ \ \ \ \ \ ⟨Δ T\ Δ V⟩\  = \ 0\)\)\)

\(TextForm\`Δ T\ Δ S\  - \ Δ p\ Δ V\  = \ \(\(C\_V\/T\) \((Δ T)\)\^2\  + \ \(\((Δ V)\)\^2\/\(V\ κ\_T\)\)\ \ \ ⟹\ \ \ ⟨Δ T\ Δ V⟩\  = \ 0\)\)

\(TextForm\`Δ T\ Δ S\  - \ Δ p\ Δ V\  = \ \(T\/C\_p\) \((Δ S)\)\^2\  + \ V\ \(κ\_S\) \((Δ p)\)\^2\)

\(TextForm\`⟨\((Δ T)\)\^2⟩\  = \ \(\(\(k\_B\) T\^2\)\/C\_V\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ⟨\((Δ V)\)\^2⟩\  = \ \(\(k\_B\) 
            T\ V\ κ\_T\ \ \ \ \ \ \ \ \ \ \ \ \ \ ⟨Δ T\ Δ V⟩\  = \ 0\)\)\)

\(TextForm\`⟨\((Δ T)\)\^2⟩\  = \ \(\(\(k\_B\) T\^2\)\/C\_V\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ⟨\((Δ V)\)\^2⟩\  = \ \(\(k\_B\) 
            T\ V\ κ\_T\ \ \ \ \ \ \ \ \ \ \ \ \ \ ⟨Δ T\ Δ V⟩\  = \ 0\)\)\)

\(TextForm\`P\  ∝ \ 
      Exp[\(-\(1\/\(2 
                   k\_B\)\)\) \((\(∂\^2 
                      S\/∂U\^2\) \((Δ E)\)\^2\  + \ 
              2 \(∂\^2 
                      S\/\(∂U ∂V\)\) Δ E\ Δ V\  + \ \(∂\^2 
                      S\/∂V\^2\) \((Δ V)\)\^2)\)]\)

\(TextForm\`P\  ∝ \ 
      Exp[\(-β\) \(1\/2\) \((\ \(∂\^2 
                      U\/∂S\^2\) \((Δ S)\)\^2\  + 
              2\ \ \((∂\^2 U\/\(∂V ∂S\))\) Δ V\ Δ S\  + \ \ \(∂\^2 
                      U\/∂V\^2\) \((Δ V)\)\^2)\)\ ]\)

\(TextForm\`P\  ∝ \ 
      Exp[\(-\(1\/\(2 
                   k\_B\)\)\) \((\(∂\^2 
                      S\/∂U\^2\) \((Δ E)\)\^2\  + \ 
              2 \(∂\^2 
                      S\/\(∂U ∂V\)\) Δ E\ Δ V\  + \ \(∂\^2 
                      S\/∂V\^2\) \((Δ V)\)\^2)\)]\)

\(TextForm\`∫\_\(-∞\)\%∞\(∫\_\(-∞\)\%∞ 
              Exp[\(-\(1\/2\)\) \((a\ x\^2\  + \ 2  b\ x\ y\  + \ 
                    c\ y\^2)\)] dx dy\)\ \  = \(\(\ \)\(2  π\)\)\/\@\(a\ c\  - \ b\^2\)\)

\(TextForm\`α\_\(i, j\)\  = \ \((∂\^2 S\/\(∂x\_i ∂x\_j\))\)\_x\&_\ \ \ \ ⟹\ \ \ S\  ≈ \ 
        S\_0\  + \ \(1\/2\) \(∑\+\(i, j\)\(α\_\(i, 
                    j\)\) Δ x\_i\ Δ x\_j\)\)

\(TextForm\`σ\_\(i, j\)\  = \ ⟨Δ x\_i\ Δ x\_j⟩\)