Boltzmann distribution, Gibbs-Shannon entropy and Helmholtz free energy

Consider a so-called canonical ensemble consisting of a system $A$ , a heat bath $R$ , and the total closed system $T$ containing $A$ and $R$ , with corresponding energies $U_A$ , $U_R$ and $U_T$ respectively so that

$U_R = U_T - U_A \quad \quad \quad \quad (1)$

with $U_T$ fixed. For example, $A$ could represent a single 1-D lattice of spins in the Ising Model, $R$ could consist of a heat bath with which $A$ is in thermal contact, and $T$ would be the closed system containing $A$ and $R$ .

With regard to the 1-D lattice of spins, different spin configurations will have different energies and for a given energy $U_A$ there will be a multiplicity of $W_A(U_A)$ spin configurations with that same energy (i.e., if $U_A$ is the macrostate, there will be $W_A(U_A)$ different microstates yielding that same macrostate). Similarly, if the energy $U_R$ is the macrostate of the heat bath, there will be $W_R(U_R)$ different microstates yielding that same macrostate. Since $U_T$ is fixed, we see that the total number of microstates for the combined system, $W_T$ , is a function of $U_A$ :

$W_T(U_A) = W_A(U_A) \times W_R(U_T - U_A) \quad \quad \quad \quad (2)$

Suppose we pick one particular spin configuration of the 1-D lattice of spins with associated energy $E_i$ . Then $U_A = E_i$ and $W_A(E_i) = 1$ because we have picked a particular microstate for system $A$ . We will now obtain from this setup the probability of the combined system being in this state. The total number of microstates for the combined system is now

$W_T(E_i) = 1 \times W_R(U_T - E_i) \quad \quad \quad \quad (3)$

The entropy of the combined system using Boltzmann’s entropy equation is

$S_R = k_B \ln(W_R) \quad \quad \quad \quad (4)$

where $k_B$ is the Boltzmann constant.

To obtain an expression for the temperature of the heat bath, we can use the first law of thermodynamics which in the absence of any work done by the system reduces to

$dU = dQ \quad \quad \quad \quad (5)$

where $dU$ is the internal energy of the system and $dQ$ is energy added via heat. Using the differential form of the definition of entropy in thermodynamics, namely

$dS = \frac{dQ}{T} \quad \quad \quad \quad (6)$

we get from (5) that

$\frac{1}{T} = \frac{dS}{dU} \quad \quad \quad \quad (7)$

Using (7) with (4) we then have

$\frac{1}{k_B T} = \frac{d \ln(W_R)}{dU_R} \quad \quad \quad \quad (8)$

Integrating gives

$W_R = \gamma e^{U_R/(k_B T)} \quad \quad \quad \quad (9)$

where $\gamma$ is an integration constant. Therefore, we have

$W_T(E_i) = W_R(U_T - E_i) = \gamma e^{(U_T - E_i)/(k_B T)} \quad \quad \quad \quad (10)$

This calculation is for one particular energy level, $U_A = E_i$ , of the 1-D lattice of spins. Summing over all possible energy levels, the total number of microstates of the combined system is

$\sum_j W_T(E_j) = \gamma e^{U_T/(k_B T)} \sum_j e^{-E_j/(k_B T)} \quad \quad \quad \quad (11)$

The probability of the combined system being in state $E_i$ is then obtained by dividing (10) by (11):

$P(E_i) = \frac{e^{-E_i/(k_B T)}}{\sum_j e^{-E_j/(k_B T)}} \quad \quad \quad \quad (12)$

This is the Boltzmann probability distribution, with

$Z = \sum_j e^{-E_j/(k_B T)} \quad \quad \quad \quad (13)$

being the partition function. Note that all terms involving the heat bath end up dropping out. The only relevance of the heat bath is to define the temperature of the system. Everything else about it is irrelevant.

The 1-D lattice of spins does not have constant energy when it is in contact with a heat bath. The energy of the system fluctuates with probabilities governed by the Boltzmann distribution. We can obtain a formula for the average entropy of the 1-D lattice of spins in the canonical ensemble, called the Gibbs-Shannon entropy, in terms of Boltzmann probabilities. Imagine taking many measurements of the energy of the 1-D lattice of spins. Interpreting $P(E_i)$ as a relative frequency, the multiplicity of microstates giving energy $E_i$ is

$W_i = \frac{1}{P(E_i)} \quad \quad \quad \quad (14)$

The entropy of the configuration giving rise to energy $E_i$ is then

$S(E_i) = k_B \ln(W_i) = -k_B \ln(P(E_i)) \quad \quad \quad \quad (15)$

The Gibbs-Shannon average entropy is obtained from (15) as

$\langle S \rangle = \sum_j P(E_i) S(E_i) = -k_B \sum_i P(E_i) \ln(P(E_i)) \quad \quad \quad \quad (16)$

Note that (16) simplifies to Boltzmann’s formula for the entropy when all the probabilities are equal, say

$P(E_i) = \frac{1}{W}$

for all $i$ , since then we have

$-k_B \sum_i P(E_i) \ln(P(E_i)) = -k_B \sum_i \frac{1}{W} \ln(\frac{1}{W}) = k_B \ln(W)$

Finally, we can obtain the Helmholtz free energy by taking logarithms of the Boltzmann probability in (12) to get

$\ln(P(E_i)) = -\frac{E_i}{k_B T} - \ln(Z) \quad \quad \quad \quad (17)$

Substituting this into the Gibbs-Shannon entropy formula in (16) we get

$\langle S \rangle = k_B \sum_i P(E_i) \bigg(\frac{E_i}{k_B T} + \ln(Z)\bigg) = \frac{\langle E \rangle}{T} + k_B \ln(Z)$

which can be rewritten as

$\langle E \rangle - T \langle S \rangle = -k_B \ln(Z) \quad \quad \quad \quad (18)$

The quantity $\langle E \rangle - T \langle S \rangle$ is the average value of the Helmholtz free energy, $F$ , which is a function of state, i.e., it is a function only of macroscopic thermodynamic variables. The equation

$F = -k_B \ln(Z) \quad \quad \quad \quad (19)$

linking the free energy to the partition function is therefore of vital importance since it relates the large-scale properties of the system to its microscopic energy states. The partition function acts like a bridge between these two regimes.

Boltzmann distribution, Gibbs-Shannon entropy and Helmholtz free energy

Published by Dr Christian P. H. Salas

Leave a comment Cancel reply