Discrete Structures for Computing
Notes 9
------------------------------------------------------------------------
Chapter 5: Counting
------------------------------------------------------------------------
This chapter studies how to COUNT objects with certain properties.
Examples:
* how many different telephone numbers can there be?
* how many different Internet addresses?
An important thing to count is the number of different PERMUTATIONS
of a set of objects -- order of the objects matters.
Ex: How many different ways are there to seat the guests at a banquet?
Another important thing to count is the number of different COMBINATIONS
of a set of objects -- order of the objects does not matter.
Ex: How many different groups of people can sit at the head table?
------------------------------------------------------------------------
5.1 The Basics of Counting
------------------------------------------------------------------------
PRODUCT RULE: Suppose there are x ways to do one task and y ways
to do another task. Then there are x*y ways to do the first task
followed by the second task.
Generalizes to more than two tasks.
Examples:
* Suppose there are 12 available offices and two students to be
assigned to office. How many ways are there to assign different
offices to the two students?
First, assign an office to the first student. There are 12 ways
to do this.
Second, assign an unoccupied office to the second student.
There are 11 ways to do this (since one office is already occupied).
So there are 12*11 = 132 ways to assign different offices to the
two students.
* How many bit strings of length 7 are there?
First, assign a bit (0 or 1) to the first bit. 2 ways to do this.
Second, assign a bit (0 or 1) to the second bit. 2 ways to do this.
...
Last, assign a bit (0 or 1) to the seventh bit. 2 ways to do this.
So there are 2*2*2*2*2*2*2 = 2^7 = 128 different bit strings of length 7.
* How many different license plates when the pattern is 3 letters followed
by 3 digits?
(26^3)*(10^3) = 17,576,000
* How many functions are there from a set with m elements to a set with
n elements?
For each of the m elements in the domain, choose one of the n elements
in the codomain to be its image.
So number of functions is n*n*...*n = n^m.
For instance, there 5*5*5 = 125 functions from {a,b,c} to {1,2,3,4,5}.
* How many one-to-one functions are there from a set with m elements to
a set with n elements?
Assume the domain is {x_1,x_2,...,x_m}.
Consider x_1 and choose one of the n elements in the codomain to be its
image.
Consider x_2 and choose one of the remaining n-1 elements in the
codomain to be its image.
Consider x_3 and choose one of the remaining n-2 elements in the
codomain to be its image.
...
Consider x_m and choose one of the remaining n - (m-1) elements in
the codomain to be its image.
So the number of such functions is n*(n-1)*(n-2)*...*(n-(m-1)).
For instance, there are 5*4*3 = 60 one-to-one functions from
{a,b,c} to {1,2,3,4,5}.
* How many different subsets are there of a set with n elements?
Use product rule to compute it.
Suppose the set is {x_1, x_2, ..., x_n}.
For each element of x, decide whether or not it is to be chosen
for the subset.
There are 2 choices for x_1 (in the subset or not in the subset).
There are 2 choices for x_2.
...
There are 2 choices for x_n.
So the total number of subsets is 2^n.
Product rule in terms of sets:
If A_1, A_2, ..., A_m are finite sets, then
|A_1 x A_2 x ... x A_m| = |A_1| * |A_2| * ... * |A_m|.
------
SUM RULE: If a task can be done either in x ways or
in y *different* ways, then there are altogether x + y different ways
to do the task.
Extends to more than two classes of ways to solve the task.
Examples:
* Suppose there are 400 computer science majors and 300 computer
engineering majors at a university and the president of the
computer game club must be either a CS or a CE major. Then
there are 400 + 300 = 700 different choices for the president.
In terms of sets: If A_1, A_2, ..., A_m are *disjoint* finite sets, then
|A_1 U A_2 U ... U A_m| = |A_1| + |A_2| + .. + |A_m|.
When the sets are not disjoint, more complicated. (More later.)
Sometimes you might need to use both the product rule and the sum rule
together.
Examples:
* How many passwords can there be in a system with these rules?
- 6 to 8 characters long
- each character is an uppercase letter or a digit
- must contain at least one digit
Total number is P6 + P7 + P8, where Pi is the number of passwords
with i characters. (Sum rule)
First, let's forget about the rule that you have to have at least
one digit. How many 6-character passwords are there?
Each of the 6 characters is either a letter or a digit, i.e., 26+10 = 36
choices (sum rule).
So the number of choices for the whole password is 36^6 (product rule).
Now let's worry about the rule that you have to have at least one digit.
total number of 6-character passwords =
number of 6-character passwords with at least one digit
+ number of 6-character passwords with no digits
How many 6-character passwords are there with no digits? 26^6.
So the number of 6-character passwords with at least one digit is
36^6 - 26^6, which is about 1.8 billion.
Similarly, the number of 7-character passwords with at least one digit is
36^7 - 26^7, which is about 70 billion.
And the number of 8-character passwords with at least one digit is
36^8 - 26^8, which is about 2.6 trillion.
So altogether, we have about 2.7 trillion.
-------
INCLUSION-EXCLUSION PRINCIPLE: Suppose a task can be done in x ways
or in y ways, but some of the x ways are the same as some of the y ways.
Sum rule doesn't work as it overcounts. Instead: Let A_1 and A_2 be
two sets (might not be disjoint). Then
|A_1 U A_2| = |A_1| + |A_2| - |A_1 /\ A_2|.
Example:
* How many bit strings of length 8 either start with 1 or end with 00?
Number that start with 1 is 1*2*2*2*2*2*2*2 = 2^7 = 128 (one choice
for first bit, 2 choices for the other bits).
Number that end with 00 is 2*2*2*2*2*2*1*1 = 2^6 = 64 (one choice for
each of the last 2 bits, 2 choices for the other bits).
We can't deduce that the number is 128 + 64, because we have
double counted those strings that start with 1 AND end with 00.
So let's find out how many they are and subtract off one copy of each.
Number that start with 1 and end with 00 is 1*2*2*2*2*2*1*1 = 2^5 = 32.
So final answer is 128 + 64 - 32 = 160.
Principle of inclusion-exclusion can be generalized to more than 2 sets.
------
Sometimes a tree diagram might be helpful to solve relatively small
problems, or to get some intuition about a problem.
<< draw figures from pp. 343-344 >>
------------------------------------------------------------------------
5.2 The Pigeonhole Principle
------------------------------------------------------------------------
A seemingly obvious, yet very useful, observation is the PIGEONHOLE
PRINCIPLE: if k+1 or more objects are placed into k boxes, then
at least one box contains more than one object.
PROOF: If every box has at most one object, then the total number of
objects would be at most k, instead of at least k+1.
QED
COROLLARY: A function from a set with m elements to a set with n
elements is not one-to-one if m > n.
PROOF: Think of each element y of the codomain as a box that holds
all elements of the domain that are mapped to y. So we have m objects
mapped to n < m boxes. By the pigeonhole principle, at least one
box (element of the codomain) has more than one object (more than one
element of the domain mapped to it).
QED
More examples of using the pigeonhole principle:
* Among any set of 367 people, at least 2 have the same birthday
(since there are only 366 different possible birthdays).
* In any group of 27 words, at least two start with the same letter
(since there are only 26 letters in the alphabet)
* There is a multiple of 6 that has only 0's and 1's in its decimal
expansion.
Proof: Look at numbers of the form 1, 11, 111, 1111, etc.
until finding two that have the same remainder when divided by 6.
1 divided by 6 has remainder 1
11 divided by 6 has remainder 5
111 divided by 6 has remainder 3
1,111 divided by 6 has remainder 1
Now look at 1,111 - 1 = 1,110.
I claim that 1,110 is a multiple of 6.
Why? 1,111 = 185*6 + 1 and 1 = 0*6 + 1.
So 1,111 - 1 = (185*6 + 1) - (0*6 + 1) = (185-0)*6.
QED
There is nothing special about 6. We can generalize this argument,
using the pigeonhole principle:
For every integer n, there is a multiple of n that has only 0's and 1's
in its decimal representation.
Proof: consider the n+1 integers in this list:
x_1 = 1
x_2 = 11
x_3 = 111
x_4 = 1,111
...
x_{n+1} = (number consisting of a string of n+1 ones)
BY THE PIGEONHOLE PRINCIPLE, there are at least two numbers in
the list that have the same remainder when divided by n (since
there are only n possible different remainders).
Call these two numbers x_i and x_j (j > i)
and let r be the common remainder.
So x_i = a*n + r for some integer a
and x_j = b*n + r for some integer b.
So x_j - x_i = (a*n + r) - (b*n + r) = (a-b)*n, which is a multiple of n.
--------
GENERALIZED PIGEONHOLE PRINCIPLE: If N objects are placed in k boxes,
then at least one box has at least ceil(N/k) objects.
Why? Proof by contradiction (notice how we work with ceiling).
Suppose N objects are placed in k boxes, but every box has less than
ceil(N/k) objects in it.
Then every box has at most ceil(N/k) - 1 objects in it.
The total number of objects is at most
k*(ceil(N/k) - 1) < k*((N/k + 1) - 1) since ceil(N/k) < N/k + 1
= k*(N/k)
= N.
I.e., the total number of objects is less than N.
But this is a contradiction to the hypothesis.
QED
Example uses of the generalized pigeonhole principle:
* Among 100 people, at least ceil(100/12) = 9 were born in the same month.
* How many cards must be drawn from a standard deck of 52 cards
to ensure at least 3 of the same suit?
One box for each suit, so k = 4.
Each card drawn is put in the appropriate box.
Need to find N such that ceil(N/4) >= 3.
Can think of worst case as when the cards drawn are distributed as evenly as
possible among the suits. Once we have drawn 2 of each suit (i.e., 2*4 = 8),
the next card drawn will cause there to be 3 of some suit.
So we have to draw 9 in the worst case.
* Show every sequence of n^2 + 1 distinct real numbers contains a
subsequence of length n+1 that is either strictly increasing or
strictly decreasing.
NOTE: subsequence does not have to be consecutive!
For instance, let n = 3. Consider this sequence of 3^2 + 1 = 10 terms:
8, 11, 9, 1, 4, 6, 12, 10, 5, 7
== = = =
Let's find a subsequence of length n+1 = 4 that is either strictly
increasing or strictly decreasing: 11, 9, 6, 5 is one.
Proof: Consider the sequence a_1, a_2, ..., a_{n^2+1}.
Let i_k be the length of the longest increasing subsequence starting at a_k.
let d_k be the length of the longest decreasing subsequence starting at a_k.
Example: a_1 = 8
a_2 = 11
a_3 = 9
a_4 = 1
a_5 = 4
Then i_1 = 2: The increasing subsequences starting at a_1 = 8 are
8
8, 11
8, 9
And d_1 = 2: The decreasing subsequences starting at a_1 = 8 are
8
8, 1
8, 4
i_2 = 1: The only increasing subsequence starting at a_2 = 11 is
11
d_2 = 3: The decreasing subsequences starting at a_2 = 11 are
11
11, 9
11, 1
11, 4
11, 9, 1
11, 9, 4
i_3 = 1: The only increasing subsequence starting at a_3 = 9 is
9
d_3 = 2: The decreasing subsequences starting at a_3 = 9 are
9
9, 1
9, 4
etc.
Suppose in contradiction there is no increasing or decreasing
subsequence of length n+1.
Then each i_k and d_k is a number between 1 and n.
By the product rule, there are n^2 different possibilities for
the combinations of i_k and d_k.
Since we have n^2 + 1 numbers in the sequence, by the pigeonhole
principle, there are two numbers in the sequence, say a_s and a_t,
that have the same value for i_s and i_t (call it x), and have the
same value for d_s and d_t (call it y).
Case 1: a_s < a_t. Then we can build an increasing sequence
starting at a_s with length x+1: start with a_s, then continue
with an increasing sequence of length x starting with a_t.
This is a contradiction to the assumption that i_s = x.
Case 2: a_s > a_t. Then we can build a decreasing sequence
starting at a_s with length y+1: start with a_s, then continue
with a decreasing sequence of length y starting with a_t.
This is a contradiction to the assumption that d_s = y.
QED
* Show that in any group of 6 people in which every 2 people
are either friends or enemies, there are 3 people who are either
all friends or all enemies.
Proof: Pick one of the people in the group, say A.
Use the generalized pigeonhole principle: there are 2 boxes,
friends of A, and enemies of A. Put the other 5 people in the appropriate
boxes. At least one box has at least ceil(5/2) = 3 people in it.
Case 1: The box with at least 3 people is the "friends of A" box.
Case 1.1: There are two people in the "friends of A" box who are
friends with each other, say B and C. Then A, B and C are all friends.
Case 1.2: All the people in the "friends of A" box are enemies with
each other. Since there are at least 3 people in this box, we have
a set of 3 people who are all enemies.
Case 2: The box with at least 3 people is the "enemies of A" box.
This case is argued analogously to Case 1.
QED
------------------------------------------------------------------------
5.3 Permutations and Combinations
------------------------------------------------------------------------
PERMUTATION of a set is an **ordered** list of the elements of the set.
Ex: S = {1,2,3}, permutations of S are
1,2,3 3,2,1 2,1,3
1,3,2 3,1,2 2,3,1
r-PERMUTATION of a set is an ordered list of r elements of the set.
Ex: The 2-permutations of S (from above) are
1,2 2,3 1,3
2,1 3,2 3,1
NOTATION: P(n,r) is the number of r-permutations of a set of n elements.
THEOREM: P(n,r) = n*(n-1)*(n-2)*...*(n-r+1).
PROOF: n ways to choose first element, n-1 ways to choose second element,
etc. Use the product rule.
QED
NOTE: P(n,0) = 1.
NOTE: P(n,r) = n!/(n-r)!
NOTE: P(n,n) = n!
EXAMPLE: How many ways are there to select first, second and third prizes
from 100 entries in a contest?
P(100,3) = 100*99*98 = 970,200
EXAMPLE: How many permutations of the letters A through H contain
A,B,C (consecutively)?
Think of having 6 objects to permute, ABC as a block and also
D, E, F, G, and H. They can occur in any order, so we have
P(6,6) = 6! = 720.
r-COMBINATION: **unordered** collection (subset) of r elements
from a set.
EXAMPLE: S = {1,2,3,4}. Then {1,3,4} is a 3-combination from S.
NOTATION: C(n,r) is number of r-combinations from a set of n elements.
Alternative notation is ( n ).
( r )
Called BINOMIAL COEFFICIENT.
EXAMPLE: S = {a, b, c, d}. Subsets of size 2 are
{a,b,}, {a,c}, {a,d}, {b,c}, {b,d}, {c,d}.
So C(4,2) = 6.
Order doesn't matter!
THEOREM: C(n,r) = n!/(r!(n-r)!).
PROOF: Think about relationship between C(n,r) and P(n,r).
P(n,r) (number of permutations, or orderings, of size r)
equals C(n,r) (number of combinations, or subsets, of size r)
times P(r,r) (number of ways to order each subst of size r).
So n!/(n-r)! = C(n,r)*r!.
Solve for C(n,r) to get the answer.
QED
EXMPLE: How many piker hands are there?
C(52,5) = 52!/(5!47!) = 52*51*50*49*49/(5*4*3*2*1) = 2,598,960
Note that C(52,47) = C(52,5).
FACT: C(n,r) = C(n,n-r).
Why? n!/(r!(n-r)!) = n!/((n-r)!(n-(n-r))!)
EXAMPLE: How many bit strings of length n contain exactly r ones?
Order doesn't matter, so position of the 1's can be considered
an r-combination. C(n,r) is the answer.
BINOMIAL THEOREM: If n is a nonnegative integer, then
(x+y)^n = Sum_{j=0}^n C(n,j)*x^{n-j}*y^j
= C(n,0)*x^n + C(n,1)*x^{n-1}*y + C(n,2)*x^{n-2}*y^2 + ...
... + C(n,n-1)*x*y^{n-1} + C(n,n)*y^n
PROOF: When you multiply out (x+y)^n, the coefficient on x^{n-j}*y^j
is what you get from adding up all the terms x^{n-j}*y^j. We have
n-j x's from the n sums and thus j y's.
QED
EXAMPLE: n = 2. (x+y)^2 = x^2 + 2xy + y^2
= C(2,0)*x^2 + C(2,1)*x*y + C(2,2)*y^2.
EXAMPLE: n = 4.
(x+y)^4 = C(4,0)*x^4 + C(4,1)*x^3*y + C(4,2)*x^2*y^2 + C(4,3)*x*y^3 +
C(4,4)*y^4
= x^4 + 4x^3*y + 6x^2*y^2 + 4xy^3 + y^4
EXAMPLE: What is the coefficient of x^12*y*13 in the expansion
of (x+y)^25?
C(25,13) = 25!/(13!12!) = 5,200,300
FACT: Sum_{k=0}^n C(n,k) = 2^n.
PROOF: 2^n = (1+1)^n
= Sum_{k=0}^n C(n,k)*1^k*1^{n-k} by Binomial Theorem
= Sum_{k=0}^n C(n,k).
An alternative proof: A set with n elements has 2^n subsets (we
showed this was true earlier in the semester). There are C(n,0) = 1
subset of size 0, C(n,1) = n subsets of size 1, C(n,2) = n(n-1)/2
subsets of size 2, ..., and C(n,n) = 1 subset of size n.
QED
FACT: Sum_{k=0}^n C(n,k)*(-1)^k = 0.
PROOF: Sum_{k=0}^n C(n,k)*(-1)^k = Sum_{k=0}^n C(n,k)*(-1)^k*1^{n-k}
= ((-1)+1)^n by the Binomial Theorem
= 0
QED
FACT: Sum_{k=0}^n C(n,k)*2^k = 3^n.
PROOF: Sum_{k=0}^n C(n,k)*2^k = Sum_{k=0}^n C(n,k)*2^k*1^{n-k}
= (2+1)^n by the Binomial Theorem
= 3^n.
QED
PASCAL'S IDENTITY: C(n+1,k) = C(n,k-1) + (n,k).
PROOF: Suppose T is a set of n+1 elements, i.e. T = {a} U S,
where S has n elements. The number of k-subsets of T is C(n+1,k).
Some of the subsets of T contain element a and some don't.
The number that don't is C(n,k) (choose k elements from the n elements
in S) .
The number that do is C(n,k-1) (choose the remaining k-1 elements
from S).
Add them together to get all the subsets of T.
QED
Pascals triangle:
<< draw Fig 1 on p. 367 >>
VANDERMONDE'S IDENTITY:
C(m+n,r) = Sum_{k=0}^r C(m,r-k)*C(n,k).
PROOF: Think of a set S that is the union of two disjoint sets T
(with m elements) and U (with n elements).
C(m+n,r) is the number of r-subsets of S.
In each r-subset of S, some of the elements come from U and some
come from T. The number of elements from U can be 0 or 1 or 2 or ...
The number of r-subsets of S with k elements from U is
C(n,k) (the number of ways to choose k elements from U) plus
C(m,r-k) (the number of ways to choose the remaining r-k elements from T).
QED
FACT: C(2n,n) = Sum_{k=0}^n C(n,k)^2.
PROOF: Follows from Vandermonde's identity with m = n.
QED
------------------------------------------------------------------------
5.5 Generalized Permutations and Combinations
------------------------------------------------------------------------
THEOREM: Number of r-permutations of a set of n objects **with
repetition allowed** is n^r.
PROOF: Product rule.
QED
EXAMPLE: Number of strings of length r from {a,b,...,z} is 26^r.
THEOREM: There are C(n+r-1,r) r-combinations from a set with
n elements when repetition is allowed.
PROOF: Think of n boxes, representing the n elements.
Then think of how many of the r chosen elements are drawn from
each box and mark with stars. Can represent like this, for
instance, with 5 boxes and 6 stars. Bars are between boxes.
This diagram:
| * * | | * * * | *
indicates choosing
0 copies of the first element
2 copies of the second element
0 copies of the third element
3 copies of the fourth element
1 copy of the fifth elemente.
In general, we have n-1 bars and r stars to arrange.
So think of n-1+r positions, to be filled with the bars and stars.
We can choose any r of the n-1+r positions for the stars, and then
insert the bars in the remaining positions. The number of ways to
choose the r positions from the stars from n-1+r available positions
is C(n-1+r,r).
QED
EXAMPLE: A bakery has 4 kinds of cookies. How many ways are
there to choose 6 cookies? Let n = 4 and r = 6 in previous theorem.
Then we have C(4+6+1,6) = C(9,6) = 84.
THEOREM: Number of permutations of n objects, with n_1 indistinguishable
objects of type 1, n_2 indistinguishable objects of type 2, ..., and
n_k indistinguishable objects of type k, is n!/(n_1!*n_2!*...*n_k!).
PROOF: The n_1 objects of type 1 can be placed in the n positions
of the overall permutation in C(n,n_1) ways, leaving n - n_1 positions
free.
The n_2 objects of type 2 can be placed in the remaining free positions
in C(n-n_1,n_2) ways.
Etc.
So we get
C(n,n_1)*C(n-n_1,n_2)*...*(Cn-n_1-...-n_{k-1},n_k).
By algebra, this is n!/(n_1!*n_2!*...*n_k!).
QED
EXAMPLE: How many different strings can be made by
reordering letters in SUCCESS?
3 S's, so let n_1 = 3
1 U, so let n_2 = 1
2 C's, so let n_3 = 2
1 E, so let n_4 = 1.
And n = 7.
So previous theorem gives us 7!/(3!*1!*2!*1!) = 420.