After using orbits and stabilizers to count how a group action moves elements of a set, we now extract a homomorphism from the action itself. Every element $g\in G$ sends each point $a\in S$ to another point $g\cdot a$, so it behaves like a permutation of $S$. This observation leads to permutation representations and extended Cayley theorem. A group action therefore converts an abstract group into a subgroup, or at least an image, inside a permutation group. In this lesson we shall prove the action-induced homomorphism, apply it to coset actions, and use the extended Cayley theorem to find normal subgroups. :::definition[Permutation Representation from an Action] Let $G$ be a group and let $S$ be a $G$-set. For each $g\in G$, define a function $\tau_g:S\to S$ by $$\begin{equation*} \tau_g(a)=g\cdot a. \end{equation*}$$ The assignment $$\begin{equation*} g\mapsto \tau_g \end{equation*}$$ is called the permutation representation associated with the action when it is viewed as a homomorphism from $G$ into the group of all permutations of $S$. ::: The action laws force each fixed group element $g$ to move the set $S$ in a reversible way. The inverse motion is supplied by $g^{-1}$, because acting by $g$ and then by $g^{-1}$ returns every point to where it started. Thus a group action does not merely give arbitrary functions $S\to S$; it gives bijections of $S$. This is why permutation representations and extended Cayley theorem connect group actions with homomorphisms into permutation groups. Students often miss the order of composition: for a left action, $\tau_{g_1g_2}=\tau_{g_1}\circ\tau_{g_2}$. :::theorem[Action-Induced Homomorphism] Let $G$ be a group and let $S$ be a $G$-set. Let $A(S)$ denote the group of all permutations of $S$. For each $g\in G$, define $\tau_g:S\to S$ by $\tau_g(a)=g\cdot a$, and define $\psi:G\to A(S)$ by $\psi(g)=\tau_g$. Then $\psi$ is a homomorphism. ::: :::proof Given that $G$ is a group, $S$ is a $G$-set, $\tau_g:S\to S$ is defined by $\tau_g(a)=g\cdot a$, and $\psi:G\to A(S)$ is defined by $\psi(g)=\tau_g$. To prove that $\psi$ is a homomorphism. [1] To prove that $\tau_g$ is a permutation of $S$. Let $a,b\in S$ and suppose $\tau_g(a)=\tau_g(b)$. Then $$\begin{equation*} g\cdot a=g\cdot b. \end{equation*}$$ Acting by $g^{-1}$ on both sides gives $$\begin{align*} g^{-1}\cdot(g\cdot a)&=g^{-1}\cdot(g\cdot b)\\ (g^{-1}g)\cdot a&=(g^{-1}g)\cdot b \quad [\because \text{ action law}]\\ e\cdot a&=e\cdot b\\ a&=b. \end{align*}$$ Therefore $\tau_g$ is one-to-one. Let $b\in S$. Put $a=g^{-1}\cdot b$. Then $$\begin{align*} \tau_g(a)&=g\cdot a\\ &=g\cdot(g^{-1}\cdot b)\\ &=(gg^{-1})\cdot b \quad [\because \text{ action law}]\\ &=e\cdot b\\ &=b. \end{align*}$$ Therefore $\tau_g$ is onto. Hence $\tau_g\in A(S)$. [2] To prove that $\psi$ preserves multiplication. Let $g_1,g_2\in G$ and let $a\in S$. Then $$\begin{align*} \tau_{g_1g_2}(a)&=(g_1g_2)\cdot a\\ &=g_1\cdot(g_2\cdot a) \quad [\because \text{ action law}]\\ &=\tau_{g_1}(\tau_{g_2}(a))\\ &=(\tau_{g_1}\circ\tau_{g_2})(a). \end{align*}$$ Thus $\tau_{g_1g_2}=\tau_{g_1}\circ\tau_{g_2}$. Hence $$\begin{equation*} \psi(g_1g_2)=\psi(g_1)\psi(g_2). \end{equation*}$$ Therefore $\psi$ is a homomorphism. $\square$ ::: :::definition[Kernel of an Action] Let $G$ act on a set $S$, and let $\psi:G\to A(S)$ be the homomorphism induced by the action. The kernel of the action is $$\begin{equation*} \ker\psi=\{g\in G:g\cdot a=a\text{ for all }a\in S\}. \end{equation*}$$ It consists of precisely those group elements that act as the identity permutation on the whole set $S$. ::: An action is called faithful when its kernel is $\{e\}$. In that case, distinct group elements induce distinct permutations of $S$, so $G$ is represented inside $A(S)$ without collapsing any non-identity element. Cayley's theorem is the most famous example: every group acts faithfully on itself by left multiplication. The extended Cayley theorem weakens this by acting on cosets of a subgroup, where the kernel may be nontrivial but is forced to lie inside that subgroup. :::theorem[Extended Cayley Theorem] Let $G$ be a group and let $H\leq G$. Let $$\begin{equation*} S=\{aH:a\in G\} \end{equation*}$$ be the set of left cosets of $H$ in $G$. Then there exists a homomorphism $$\begin{equation*} \psi:G\to A(S) \end{equation*}$$ such that $$\begin{equation*} \ker\psi\subseteq H. \end{equation*}$$ ::: :::proof Given that $G$ is a group, $H\leq G$, and $S=\{aH:a\in G\}$ is the set of left cosets of $H$ in $G$. To prove that there exists a homomorphism $\psi:G\to A(S)$ such that $\ker\psi\subseteq H$. Define an action of $G$ on $S$ by $$\begin{equation*} g\cdot(aH)=(ga)H. \end{equation*}$$ This is well-defined because left multiplication sends a left coset to a left coset. The identity element satisfies $$\begin{equation*} e\cdot(aH)=(ea)H=aH. \end{equation*}$$ For $g_1,g_2\in G$ and $aH\in S$, $$\begin{align*} (g_1g_2)\cdot(aH)&=((g_1g_2)a)H\\ &=(g_1(g_2a))H\\ &=g_1\cdot((g_2a)H)\\ &=g_1\cdot(g_2\cdot(aH)). \end{align*}$$ Thus $G$ acts on $S$. By the action-induced homomorphism theorem, this action gives a homomorphism $$\begin{equation*} \psi:G\to A(S). \end{equation*}$$ Let $g\in\ker\psi$. Then $g$ fixes every left coset in $S$. In particular, it fixes $eH=H$, so $$\begin{equation*} gH=H. \end{equation*}$$ Therefore $g\in H$. Thus $\ker\psi\subseteq H$. Hence there exists a homomorphism $\psi:G\to A(S)$ such that $\ker\psi\subseteq H$. $\square$ ::: :::corollary Let $G$ be a finite group and let $H$ be a proper subgroup of $G$ of index $n$. If $|G|$ does not divide $n!$, then $G$ contains a nontrivial normal subgroup. ::: :::proof Given that $G$ is a finite group, $H$ is a proper subgroup of $G$, $[G:H]=n$, and $|G|$ does not divide $n!$. To prove that $G$ contains a nontrivial normal subgroup. Let $S=\{aH:a\in G\}$. Then $|S|=n$. By the extended Cayley theorem, there exists a homomorphism $$\begin{equation*} \psi:G\to A(S) \end{equation*}$$ such that $\ker\psi\subseteq H$. Since $|S|=n$, the group $A(S)$ has order $n!$. Also $\psi(G)\leq A(S)$, so $|\psi(G)|$ divides $n!$. By the first isomorphism theorem, $$\begin{equation*} G/\ker\psi\cong \psi(G). \end{equation*}$$ If possible let $\ker\psi=\{e\}$. Then $$\begin{equation*} |G/\ker\psi|=|G|. \end{equation*}$$ Thus $|G|=|\psi(G)|$, and hence $|G|$ divides $n!$. This contradicts the hypothesis. Therefore $\ker\psi\neq\{e\}$. Since kernels of homomorphisms are normal subgroups, $\ker\psi\trianglelefteq G$. Hence $G$ contains a nontrivial normal subgroup. $\square$ ::: Use this calculator for the common finite-group application of the extended Cayley theorem. Enter the order $|G|$ and the index $n=[G:H]$. If $|G|$ does not divide $n!$, then the coset action cannot be faithful, so its kernel gives a nontrivial normal subgroup inside $G$. This test does not prove that $H$ itself is normal; it detects a normal subgroup contained in $H$. :::calculator[Extended Cayley Divisibility Test] @libraries: none ::: :::solved-problem Let $G$ be a finite group and let $H\leq G$ be a subgroup of index $p$, where $p$ is the smallest prime dividing $|G|$. Prove that $H\trianglelefteq G$. ::: :::solution Let $G$ be a finite group and let $H\leq G$ such that $[G:H]=p$, where $p$ is the smallest prime dividing $|G|$. Given that $p$ is the smallest prime divisor of $|G|$. To prove that $H\trianglelefteq G$. Let $S=\{aH:a\in G\}$. Then $|S|=p$. The coset action gives a homomorphism $$\begin{equation*} \psi:G\to A(S) \end{equation*}$$ with $\ker\psi\subseteq H$. Since $|S|=p$, the group $A(S)$ has order $p!$. By the first isomorphism theorem, $G/\ker\psi$ is isomorphic to a subgroup of $A(S)$. Hence $$\begin{equation*} |G/\ker\psi|\mid p!. \end{equation*}$$ Since $\ker\psi\subseteq H$, we have $$\begin{align*} |G/\ker\psi|&=[G:\ker\psi]\\ &=[G:H][H:\ker\psi]\\ &=p[H:\ker\psi]. \end{align*}$$ Thus $p[H:\ker\psi]$ divides $p!$. If $[H:\ker\psi]>1$, then it has a prime divisor $q$. Since $[H:\ker\psi]$ divides $|H|$, the prime $q$ divides $|G|$. By minimality of $p$, we have $q\geq p$. But $p[H:\ker\psi]\mid p!$ allows no additional prime factor at least $p$ beyond the single factor $p$ already present. A contradiction. Therefore $[H:\ker\psi]=1$, so $H=\ker\psi$. Since kernels of homomorphisms are normal subgroups, $H\trianglelefteq G$. $\square$ ::: :::solved-problem Let $H$ be a subgroup of order $11$ and index $4$ in a finite group $G$. Prove that $H\trianglelefteq G$. ::: :::solution Let $H$ be a subgroup of a finite group $G$ such that $|H|=11$ and $[G:H]=4$. Given that $|H|=11$ and $[G:H]=4$. To prove that $H\trianglelefteq G$. Since $[G:H]=4$, we get $$\begin{equation*} |G|=4\cdot 11=44. \end{equation*}$$ Let $G$ act on the set of left cosets of $H$. This gives a homomorphism $$\begin{equation*} \psi:G\to S_4 \end{equation*}$$ such that $\ker\psi\subseteq H$. Since $|S_4|=4!=24$ and $44$ does not divide $24$, the kernel cannot be trivial. Therefore $$\begin{equation*} \ker\psi\neq\{e\}. \end{equation*}$$ Since $\ker\psi\subseteq H$ and $|H|=11$, the only subgroups of $H$ are $\{e\}$ and $H$. Thus $$\begin{equation*} \ker\psi=H. \end{equation*}$$ Since kernels of homomorphisms are normal subgroups, $H\trianglelefteq G$. $\square$ ::: :::tip In a coset action, the kernel is not an arbitrary subgroup. It is a normal subgroup of $G$ and it lies inside $H$. This is why the extended Cayley theorem is so useful for proving non-simplicity: a nontrivial kernel immediately supplies a nontrivial normal subgroup. ::: :::exercise 1. Let $G$ act on itself by left multiplication. Prove that the induced homomorphism $G\to A(G)$ has trivial kernel. 2. Let $G$ be a finite group of order $80$ with a subgroup $H$ of order $16$. Use the extended Cayley theorem to show that $G$ is not simple. 3. Let $H\leq G$ have index $n$. Explain why the coset action gives a homomorphism from $G$ into $S_n$ when $G$ is finite. ::: :::answer 1. If $g$ is in the kernel, then $g\cdot a=a$ for all $a\in G$. Taking $a=e$ gives $g=e$, so the kernel is $\{e\}$. 2. Here $[G:H]=80/16=5$. The coset action gives $\psi:G\to S_5$ with $\ker\psi\subseteq H$. Since $80$ does not divide $5!=120$, the kernel is nontrivial. Thus $G$ has a nontrivial normal subgroup and is not simple. 3. The set of left cosets has $n$ elements, so its full permutation group is isomorphic to $S_n$. The action-induced homomorphism maps $G$ into that permutation group. ::: :::faq[Frequently Asked Questions] Q: Is every action-induced homomorphism one-to-one? A: No. It is one-to-one exactly when the action is faithful, meaning the kernel is $\{e\}$. Q: Why does the extended Cayley theorem use left cosets? A: Left multiplication by group elements naturally sends left cosets to left cosets, producing a well-defined action. Q: Does $\ker\psi\subseteq H$ imply $H$ is normal? A: Not by itself. If the argument further proves $\ker\psi=H$, then $H$ is normal because kernels are normal. ::: A permutation representation records how a single group element moves every point of the acted-upon set. Use the inputs to preview the action of an element of $\mathbb{Z}_n$ on $\mathbb{Z}_n$ by addition modulo $n$. The output lists the induced permutation $\tau_g$ as a mapping table. Notice that only the identity element induces the identity permutation in this regular action, which is why this action is faithful. :::scientific-preview[Action Induced Permutation Preview] @libraries: none ::: :::call-to-action[Continue to Fixed Points and Burnside's Lemma] subtitle: Learn how fixed points average over a finite group action to count the number of orbits. button: Study Burnside's Lemma | /abstract-algebra/homomorphisms-and-isomorphisms-of-groups/group-actions/fixed-points-and-burnsides-lemma button-ghost: Review Orbit Counting | /abstract-algebra/homomorphisms-and-isomorphisms-of-groups/group-actions/orbit-stabilizer-formula-and-orbit-counting :::