We know that \[\lim_{n \to \infty}\left(1+\frac{x}{n}\right)^n=\sum_{n=0}^{\infty}\frac{x^n}{n!}=e^x \quad \text{for any} \quad x \in\mathbb{R}.\]
Exponential function. The function \(E:\mathbb{R} \to (0,\infty)\) defined by \(E(x)=e^x\) is called the exponential function.
Properties of exponential function.
For all \(x,y \in \mathbb{R}\) one has \[e^{x+y}=e^xe^y.\]
If \(\lim_{n \to \infty}a_n = a\), then \(\lim_{n \to \infty}e^{a_n}=e^{a}\).
\(E\) is one-to-one and onto. Thus the inverse for \(E\) exists.
Natural logarithm. The inverse of \(E\) exists. It will be denoted by \(E^{-1}:(0,\infty) \to \mathbb{R}\), \[E^{-1}(x)=\ln(x)=\log(x)\] and it is called the natural logarithm.
Simple properties of natural logarithm.
\(\log(x)\) is increasing.
For \(x,y \in (0,\infty)\) we have \[\log(xy)=\log(x)+\log(y).\]
We also have \(x^{\alpha}=e^{\alpha \log(x)}\) for all \(\alpha \in \mathbb{R}\).
Proposition. For \(x>0\) we have \[\frac{x}{x+2}<\log(x+1)<x.\]
Proof. We prove that for \(0<x<m\) with \(m \in \mathbb{N}\), we have \[\left(1+\frac{x}{n}\right)^n<e^x<\left(1+\frac{x}{n}\right)^{n+m}.\]
thus \[n\log\left(1+\frac{x}{n}\right)<x<(n + m)\log\left(1+\frac{x}{n}\right).\]
Hence \[\frac{x}{n+m}<\log\left(1+\frac{x}{n}\right)<\frac{x}{n} \quad \text{ if } \quad m>x.\]
Taking \(n=1\) we obtain \[\log(1+x)<x \quad \text{ for all } \quad x>0.\]
Now set \(m=\lfloor x\rfloor+1>x\), then \[\log\left(1+\frac{x}{n}\right)>\frac{\frac{x}{n}}{2+\frac{x}{n}}.\]
Thus for \(n=1\) we obtain \[\log(1+x)>\frac{x}{2+x}.\] $$\tag*{$\blacksquare$}$$
Remark. In fact, for every \(x>0\) the following inequality holds \[\frac{x}{x+1}<\log(x+1)<x.\]
Divergence of harmonic series. \[\sum_{n=1}^{\infty}\frac{1}{n}=+\infty.\]
Theorem. The sequences \[{\color{blue}a_n=\sum_{k=1}^{n-1}\frac{1}{k}-\log(n) \quad \text{ and } \quad b_n=\sum_{k=1}^{n}\frac{1}{k}-\log(n)}\] are increasing and decreasing respectively and bounded, and \[\lim_{n \to \infty}a_n=\lim_{n \to \infty}b_n=\gamma.\] where \(\gamma\) is known as the Euler (or Euler–Mascheroni) constant.
Remark.
It is not even known whether \(\gamma\) is irrational.
\(\gamma\) is called Euler-Mascheroni constant, and \(\gamma \simeq 0,5772\ldots\).
Proof. We know \[\left(1+\frac{1}{n}\right)^n<e<\left(1+\frac{1}{n}\right)^{n+1}\] thus \[n \log\left(1+\frac{1}{n}\right)<1<(n+1)\log\left(1+\frac{1}{n}\right),\] and consequently \[\log\left(\frac{n+1}{n}\right)<\frac{1}{n},\] \[\log\left(\frac{n+1}{n}\right)>\frac{1}{n+1}.\]
Thus \[\begin{aligned} a_{n+1}-a_n=\sum_{k=1}^{n}\frac{1}{k}-\log(n+1)-\sum_{k=1}^{n-1}\frac{1}{k}+\log(n)= \frac{1}{n}-\log\left(\frac{n+1}{n}\right)>0. \end{aligned}\] Hence \((a_n)_{n\in\mathbb N}\) is increasing. Similarly,
\[\begin{aligned} b_{n+1}-b_n=\frac{1}{n+1}-\log\left(\frac{n+1}{n}\right)<0, \end{aligned}\] thus \((b_n)_{n\in\mathbb N}\) is decreasing. Also it is clear \[a_1 \leq a_n \leq b_n \leq b_1.\] Thus by the (MCT) the limits exist \[\lim_{n \to \infty}a_n=\lim_{n \to \infty}b_n=\gamma,\] since \(b_n=a_n+\frac{1}{n}\).$$\tag*{$\blacksquare$}$$
Upper limit and lower limit. Let \((s_n)_{n \in \mathbb{N}}\) be a sequence of real numbers.
The upper limit is defined by \[\limsup_{n \to \infty}s_n=\inf_{k \geq 1}\sup_{n \geq k}s_n.\]
The lower limit is defined by \[\liminf_{n \to \infty}s_n=\sup_{k \geq 1}\inf_{n \geq k}s_n.\]
Proposition. For a sequence \((s_n)_{n \in \mathbb{N}}\subset\mathbb R\), the upper and lower limits always exist.
Proof. Let \(\alpha_k=\sup_{n \geq k}s_n\). Then \({\color{blue}\alpha_{k+1} \leq \alpha_k}\) and \[\limsup_{n \to \infty}s_{n}=\inf_{k \geq 1}\sup_{n \geq k}s_n \underbrace{=}_{{\color{blue}(MCT)}}\lim_{n \to \infty}\alpha_k \text{ {\color{red} (possible infinite!)}}.\]
If \(\beta_k=\inf_{n \geq k}s_n\), then \({\color{blue}\beta_k \leq \beta_{k+1}}\) and \[\liminf_{n \to \infty}s_{n}=\sup_{k \geq 1}\inf_{n \geq k}s_n \underbrace{=}_{{\color{blue}(MCT)}}\lim_{n \to \infty}\beta_k \text{ {\color{red} (possible infinite!)}}.\] $$\tag*{$\blacksquare$}$$
Useful remarks. We always have \[\beta_k=\inf_{n \geq k}s_n \leq \sup_{n \geq k}s_{n}=\alpha_k.\] thus \[{\color{red}\liminf_{n \to \infty}a_n=\lim_{k \to \infty }\beta_k \leq \lim_{k \to \infty}\alpha_k = \limsup_{n \to \infty}s_n}.\]
Proposition. If \(\liminf_{n \to \infty}s_n=\limsup_{n \to \infty}s_n=L\) then \(\lim_{n\to\infty}s_n=L\).
Proof. If \(\liminf_{n \to \infty}s_n=\limsup_{n \to \infty}s_n=L,\) then \[\alpha_k=\inf_{n \geq k}s_n \leq s_k \leq \sup_{n \geq k}s_n=\beta_k\]
and \(\lim_{k \to \infty}\alpha_k=\lim_{k \to \infty}\beta_k=L\), thus \(\lim_{n \to \infty}s_n=L\). $$\tag*{$\blacksquare$}$$
Example 1. Consider \(a_n=(-1)^n\frac{n+1}{n}\). Let \[\beta_n=\sup\left\{(-1)^n\frac{n+1}{n},(-1)^{n+1}\frac{n+2}{n+1},\ldots\right\},\] then \[\beta_n=\begin{cases}\frac{n+1}{n} \text{ if }n \text{ is even,}\\ \frac{n+2}{n+1} \text{ if }n \text{ is odd.} \end{cases}\]
Thus \(\lim_{n \to \infty}\beta_n=1\). Therefore \[\limsup_{n \to \infty}a_n=1.\] Similarly \[\liminf_{n \to \infty}a_n=-1.\]
Example 2. Let \[a_n=\begin{cases}0 \text{ if }n \text{ is odd},\\ 1 \text{ if }n \text{ is even.} \end{cases}\] Then \[\begin{aligned} \beta_n&=\sup\left\{a_m\;:\;m \geq n\right\}=1, \\ \alpha_n&=\inf\{a_m\;:\;m \geq n\}=0. \end{aligned}\]
Therefore \[\begin{aligned} \limsup_{n \to \infty}a_n&=1,\\ \liminf_{n \to \infty}a_n&=0. \end{aligned}\]
Example 3. Let \(a_n=\frac{1}{n}\). Then \[\beta_n=\sup\left\{\frac{1}{m}\;:\;m \geq n\right\}=\frac{1}{n},\] so \(\lim_{n \to \infty}\beta_n=0.\) Similarly \[\alpha_n=\inf\left\{\frac{1}{m}\;:\;m \geq n\right\}=0,\] so \(\lim_{n \to \infty}\alpha_n=0\). Thus \[\limsup_{n \to \infty}a_n=\liminf_{n \to \infty}a_ n=0.\]
Absolute convergence. The series \(\sum_{n=1}^{\infty}a_n\) is said to converge absolutely if the series \[\sum_{n=1}^{\infty}|a_n|<\infty\] converges.
Theorem. If \(\sum_{n=1}^{\infty}|a_n|<\infty\), then \(\left|\sum_{n=1}^{\infty}a_n\right|<\infty\).
Proof. The claim follows from the Cauchy Criterion, since \[\left|\sum_{k=m}^{n}a_k\right| \leq \sum_{k=m}^{n}|a_k|\] and we are done. $$\tag*{$\blacksquare$}$$
Conditional convergence. If the series \(\sum_{n=1}^{\infty}a_n\) converges but \[\sum_{n=1}^{\infty}|a_n|=\infty\] diverges then we say that \(\sum_{n=1}^{\infty}a_n\) converges conditionally.
Example 1. For series with positive terms, absolute convergence is the same as convergence.
Example 2. \(\sum_{n=1}^{\infty}(-1)^n\frac{1}{n^2}\) converges absolutely, since \(\sum_{n=1}^{\infty}\frac{1}{n^2}<\infty\).
Anharmonic series. The series \[\sum_{n=1}^{\infty}\frac{(-1)^n}{n}\] converges conditionally.
It is easy to see that \[\sum_{n=1}^{\infty}\left|\frac{(-1)^n}{n}\right|=\sum_{n=1}^{\infty}\frac{1}{n}=\infty.\] To prove \(\big|\sum_{n=1}^{\infty}\frac{(-1)^n}{n}\big|<\infty\) we will show a more general result.
Abel summation formula. Given two sequences \((a_n)_{n \in \mathbb{N}}\) and \((b_n)_{n \in \mathbb{N}}\) set \[A_n=\sum_{k=0}^{n}a_k \quad \text{ for } \quad n \geq 0, \quad \text{ and } \quad A_{-1}=0.\]
Then if \(0 \leq p \leq q\) one has \[{\color{blue}\sum_{n=p}^{q}a_nb_n=\sum_{n=p}^{q-1}A_n \left(b_n-b_{n+1}\right)+A_qb_q-A_{p-1}b_p}.\]
Proof: Note that \[\begin{aligned} \sum_{n=p}^{q}a_nb_n&= \sum_{n=p}^q\underbrace{(A_n-A_{n-1})}_{{\color{red}a_n}}b_n= \sum_{n=p}^{q}A_nb_n-\sum_{n=p}^q A_{n-1}b_n \\ &= \sum_{n=p}^{q}A_nb_n-\sum_{n=p-1}^{q-1} A_{n}b_{n+1} \\&=\sum_{n=p}^{q-1}A_n \left(b_n-b_{n+1}\right)+A_qb_q-A_{p-1}b_p. \end{aligned}\] The proof follows. $$\tag*{$\blacksquare$}$$
Dirichlet’s test. Suppose that
The partial sums \(A_n=\sum_{k=1}^na_k\) of \((a_n)_{n\in \mathbb{N}}\) form a bounded sequence.
\(b_0 \geq b_1 \geq b_2 \geq b_3 \geq \ldots\),
\(\lim_{n \to \infty}b_n=0.\)
Then \(\sum_{n=1}^{\infty}a_nb_n\) converges.
Proof. Choose \(M \geq 0\) so that \(|A_n| \leq M\) for all \(n \in \mathbb{N}\). Given \(\varepsilon>0\) there is \(N_{\varepsilon} \in \mathbb{N}\) so that \[b_{N_{\varepsilon}}<\frac{\varepsilon}{2M},\] since \(\lim_{n \to \infty}b_n=0\).
For \(N_{\varepsilon} \leq p \leq q\), by the summation by parts formula, one has \[\begin{aligned} \left|\sum_{n=p}^q a_nb_n\right|&=\left|\sum_{n=p}^{q-1}A_n \left(b_n-b_{n+1}\right)+A_qb_q-A_{p-1}b_p\right| \\&\leq {\color{red}\left|\sum_{n=p}^{q-1}A_n \left(b_n-b_{n+1}\right)\right|}+{\color{blue}|A_qb_q|}+{\color{purple}|A_{p-1}b_p|} \\&\leq M {\color{red}\sum_{n=p}^{q-1}\left| \left(b_n-b_{n+1}\right)\right|}+M{\color{blue}b_q}+M{\color{purple}b_p} \leq 2Mb_p \leq 2Mb_{N_{\varepsilon}}<\varepsilon. \end{aligned}\] since \[\begin{aligned} b_p&-b_q={\color{red}\sum_{n=p}^{q-1}\left| \left(b_n-b_{n+1}\right)\right| }=\sum_{n=p}^{q-1} \left(b_n-b_{n+1}\right) \\ &=(b_p-{\color{red}b_{p+1}})+({\color{red}b_{p+1}}-{\color{blue}b_{p+2}})+({\color{blue}b_{p+2}}-{\color{purple}b_{p+3}})+\ldots+b_{q-1}-b_q.\end{aligned}\qquad\blacksquare\]
We now show that \(\sum_{n=1}^{\infty}\frac{(-1)^n}{n}\) converges. Let \[{\color{red}a_n=(-1)^n,} \quad \text{ and }\quad {\color{blue}b_n=\frac{1}{n}}\] in the previous theorem. We see that \[\left|\sum_{n=1}^{\infty}\frac{(-1)^n}{n}\right|=\left|\sum_{n=1}^{\infty}a_nb_n\right|<\infty\] since \[|A_n|=\left|\sum_{k=1}^n(-1)^k\right| \leq 1.\]
A more general result can be proved:
Alternating Series Test. Let \((a_n)_{n \in \mathbb{N}}\) be such that
\(a_1 \geq a_2 \geq \ldots \geq a_n \geq \ldots\),
\(\lim_{n \to \infty}a_n=0\).
Then the alternating series \(\sum_{n=1}^{\infty}(-1)^n a_n\) converges.
Proof. We apply the previous theorem.
Exercise. Determine if the series \(\sum_{n=1}^{\infty}\frac{(-1)^n}{\sqrt{n^2+1}}\) converges and converges absolutely.
Solution. Let \(a_n=\frac{1}{\sqrt{n^2+1}}\).
We have \[a_n \geq \frac{1}{\sqrt{4n^2}}=\frac{1}{2n},\] so the series does not converges absolutely, since \(\sum_{n=1}^{\infty}\frac{1}{n}\) diverges.
On the other hand, we have \[a_n \geq a_{n+1} \quad \text{ and }\quad \lim_{n \to \infty}a_n=0,\] so the assumptions of the previous theorem are satisfied. Hence \(\sum_{n=1}^{\infty}\frac{(-1)^n}{\sqrt{n^2+1}}\) converges conditionally.$$\tag*{$\blacksquare$}$$
Root test. Given \(\sum_{n=1}^{\infty}a_n\) set \[{\color{blue}\alpha=\limsup_{n \to \infty}\sqrt[n]{|a_n|}}.\]
If \(\alpha<1\), then \(\sum_{n=1}^{\infty}a_n\) converges.
If \(\alpha>1\), then \(\sum_{n=1}^{\infty}a_n\) diverges.
If \(\alpha=1\), no information.
Proof. If \(\alpha<1\) we can choose \(\beta\) so that \(\alpha<\beta<1\) and the integer \(N \in \mathbb{N}\) so that \[\sqrt[n]{|a_n|}<\beta \quad \text{ for all }\quad n \geq N,\] since \[\alpha=\inf_{k \geq 1}\sup_{n \geq k}\sqrt[n]{|a_n|}<\beta.\]
For \(n \geq N\) we have \(|a_n|<\beta^n\), but \(\beta<1\), thus \(\sum_{n=1}^{\infty}\beta^n\) converges and the comparision test implies that \(\sum_{n=1}^{\infty}a_n\) converges as well.
If \(\alpha>1\) then there is \((n_k)_{k \in \mathbb{N}}\) so that \[{|a_{n_k}|}^{1/n_k} \ _{\overrightarrow{k \to \infty}}\ \alpha.\]
Hence \(|a_n|>1\) holds for infinitely many values of \(n \in \mathbb{N}\), so that the condition \(a_n \ _{\overrightarrow{n \to \infty}}\ 0\) necessary for convergence \(\sum_{n=1}^{\infty}a_n\) does not hold.
To prove (c) note that \[\sum_{n=1}^{\infty}\frac{1}{n}=\infty \quad \text{ and } \quad \sqrt[n]{n} \ _{\overrightarrow{n \to \infty}}\ 1.\] \[\sum_{n=1}^{\infty}\frac{1}{n^2}<\infty \quad \text{ and }\quad \sqrt[n]{n^2} \ _{\overrightarrow{n \to \infty}}\ 1.\] This completes the proof. $$\tag*{$\blacksquare$}$$
Example 1. \[\sum_{n=1}^{\infty}\left(\frac{e}{n}\right)^n<\infty,\] since \[\sqrt[n]{\frac{e^n}{n^n}}=\frac{e}{n}\ _{\overrightarrow{n \to \infty}}\ 0.\]
Example 2. \[\sum_{n=1}^{\infty}\frac{n^2}{2^n}<\infty,\] since \[\sqrt[n]{\frac{n^2}{2^n}}\ _{\overrightarrow{n \to \infty}}\ \frac{1}{2}.\]
Ratio test. The series \(\sum_{n=1}^{\infty}a_n\)
converges if \(\limsup_{n \to \infty}\left|\frac{a_{n+1}}{a_n}\right|<1\),
diverges if \(\left|\frac{a_{n+1}}{a_n}\right| > 1\) for all \(n \geq n_0\) for some fixed \(n_0 \in \mathbb{N}\).
Proof. If (a) holds we can find \(\beta<1\) and \(n \in \mathbb{N}\) such that \[\left|\frac{a_{n+1}}{a_n}\right|<\beta \quad \text{ for all }\quad n \geq N.\]
In particular, for \(p \in \mathbb{N}\), one has \[\begin{aligned} |a_{n+p}|&=|a_{n+p-1}|\frac{|a_{n+p}|}{|a_{n+p-1}|}\\ &<\beta |a_{n+p-1}| \\&<\beta^2 |a_{n+p-2}|<\ldots< \\ &<\beta^p|a_n|. \end{aligned}\]
Thus \(|a_{N+p}|<\beta^p|a_N|\) and \[|a_n|<|a_N|\beta^{-N}\beta^n \quad \text{ for all } \quad n \geq N.\]
The claim follows from the comparison test since \(\sum_{n=1}^{\infty}\beta^n<\infty\) whenever \(\beta<1\).
If \(|a_{n+1}| \geq |a_n|\) for \(n \geq n_0\) then \(a_n \ _{\overrightarrow{n \to \infty}}\ 0\) does not hold. $$\tag*{$\blacksquare$}$$
Remark. As before \(\lim_{n \to \infty}\frac{a_{n+1}}{a_n}=1\) is useless: \[{\color{red}\sum_{n=1}^{\infty}\frac{1}{n}=\infty\quad \text{ and } \quad \frac{a_{n+1}}{a_n}=\frac{n}{n+1} \ _{\overrightarrow{n \to \infty}}\ 1},\] \[{\color{blue}\sum_{n=1}^{\infty}\frac{1}{n^2}<\infty\quad \text{ and }\quad \frac{a_{n+1}}{a_n}=\left(\frac{n}{n+1}\right)^2 \ _{\overrightarrow{n \to \infty}}\ 1.}\]
Example. \(\sum_{n=1}^{\infty}\frac{n!}{n^n}<\infty\), since \[\begin{aligned} \frac{a_{n+1}}{a_n}= \frac{(n+1)!}{(n+1)^{n+1}}\frac{n^n}{n!}=\frac{(n+1)n^n}{(n+1)^{n+1}}=\left(\frac{n}{n+1}\right)^n \ _{\overrightarrow{n \to \infty}}\ \frac{1}{e}<1. \end{aligned}\]
Rearrangement. Let \((k_n)_{n \in \mathbb{N}}\) be a sequence in which every positive integer appears once and only once. Setting \[{\color{blue}a_n'=a_{k_n}}\] we say that \(\sum_{n=1}^{\infty}a_n'\) is rearrangement of \(\sum_{n=1}^{\infty}a_n\).
Example.
Consider the convergent series \[S=\sum_{n=1}^{\infty}\frac{(-1)^{n+1}}{n}=1-\frac{1}{2}+\frac{1}{3}\underbrace{-\frac{1}{4}+\frac{1}{5}}_{{\color{blue} <0}}\underbrace{-\frac{1}{6}+\frac{1}{7}}_{{\color{blue}<0}}-\ldots\]
Consider also a rearrangement \(S'\) of \(S\) given by: \[\begin{aligned} S'&=1+\frac{1}{3}-\frac{1}{2}+\frac{1}{5}+\frac{1}{7}-\frac{1}{4}+\frac{1}{9}+\frac{1}{11}-\frac{1}{6}+\ldots+\\ &=\sum_{k=1}^{\infty}\bigg(\frac{1}{4k-3}+\frac{1}{4k-1}-\frac{1}{2k}\bigg) \end{aligned}\]
Observe that \(S<1-\frac{1}{2}+\frac{1}{3}=\frac{5}{6}\) and \[\frac{1}{4k-3}+\frac{1}{4k-1}-\frac{1}{2k}>0 \quad \text{ for all } \quad k \in \mathbb{N}.\]
If \(S_n'\) is the partial sum of \(S'\) then \[S_3'<S_6'<S_9'<\ldots\] hence \(\limsup_{n \to \infty}S_n'>S_3'=\frac{5}{6}\).
Thus \(S'\) does not converge to \(S<\frac{5}{6}\).
Theorem. Let \(\sum_{n=1}^{\infty}a_n\) be a series that converges unconditionally. Suppose that \[-\infty \leq {\color{red}\alpha} \leq {\color{blue}\beta} \leq +\infty.\]
Then there exists a reaarangement \(\sum_{n=0}^{\infty}a_n'\) with partial sums \(s_n'\) so that \[{\color{red}\liminf_{n \to \infty}s_n'=\alpha}, \quad \text{ and } \quad {\color{blue}\limsup_{n \to \infty}s'_n=\beta}.\]
Theorem. If \(\sum_{n=1}^{\infty}|a_n|<\infty\), then every rearrangement of \(\sum_{n=1}^{\infty}a_n\) converge to the same limit.
Proof. Let \(\sum_{n=1}^{\infty}a_n'\) be a rearrangement of \(\sum_{n=1}^{\infty}a_n\) with partial sums \(s_n'\). Given \(\varepsilon>0\) there is \(N_{\varepsilon} \in \mathbb{N}\) such that \(m \geq n \geq N_{\varepsilon}\) implies \[\left|\sum_{k=n}^ma_k\right|<\varepsilon.\]
Now choose \(p \in \mathbb{N}\) such that \[\{1,2,\ldots,N_{\varepsilon}\} \subseteq \{k_1,k_2,\ldots,k_p\}.\] If \(n>p\) then the numbers \(a_1,\ldots,a_N\) will cancel in the difference \(s_n-s_n'\) so that
\[|s_n-s_n'|<\varepsilon.\]
Hence \(s_n'\) converges to the same limit as \((s_n)_{n \in \mathbb{N}}\).$$\tag*{$\blacksquare$}$$