Homework #10

EE 503: Fall 2025

Assignment Details

Assigned: 28 October
Due: Tuesday, 04 November at 12:00

BrightSpace Assignment: Submit your solutions to BrightSpace

Instructions

Write your solutions to these homework problems on separate sheets of paper. Submit your work to Brightspace by the due date. Show all work and box answers where appropriate. Do not guess.

Problem 1

Suppose random variable $X$ has a discrete probability density $P(X) = (\frac{1}{2}, \frac{1}{4}, \frac{1}{8}, \frac{1}{8})$ and random variable $Y$ has density $P(Y) = (\frac{1}{4}, \frac{1}{4}, \frac{1}{4}, \frac{1}{4})$ and both have joint density $P(X,Y)$ where the element in the $i$th row and $j$th column is $p(x_i, y_j)$:

\[P(X, Y) = \begin{pmatrix} \frac{1}{8} & \frac{1}{16} & \frac{1}{16} & \frac{1}{4} \\ \frac{1}{16} & \frac{1}{8} & \frac{1}{16} & 0 \\ \frac{1}{32} & \frac{1}{32} & \frac{1}{16} & 0 \\ \frac{1}{32} & \frac{1}{32} & \frac{1}{16} & 0 \end{pmatrix}.\]

Define the entropy of $X$ in bits as $H(X) = -\sum_{i=1}^{n} p_i \log_2 p_i$, the conditional entropy $H(X|Y) = -\sum_{i=1}^{n} \sum_{j=1}^{n} p(x_i, y_j) \log_2 p(x_i | y_j)$, and the joint entropy as $H(X,Y) = -\sum_{i=1}^{n} \sum_{j=1}^{n} p(x_i, y_j) \log_2 p(x_i, y_j)$. Define the mutual information $I(X,Y) = \sum_{i=1}^{n} \sum_{j=1}^{n} p(x_i, y_j) \log_2 \frac{p(x_i,y_j)}{p(x_i) p(y_j)}$. Then $I(X,Y) \ge 0$ holds. First compute $H(X)$, $H(Y)$, $H(X,Y)$, $H(X|Y)$, $H(Y|X)$, and $I(X,Y)$ for the above densities. Then prove that $H(X,Y) = H(X) + H(Y|X)$ and $I(X,Y) = H(X) - H(X|Y)$ and verify for the above densities. Thus $I(X,Y) = H(X) + H(Y) - H(X,Y)$. Verify this too.

Problem 2

A subset $A \subset X$ is fuzzy iff it has a multivalues indicator function $I_A: X \rightarrow [0,1]$ or more simply $a: X \rightarrow [0,1]$. So a finite fuzzy set $A$ defines a point in the $n$-dimensional unit hypercube: $A \subset [0,1]^n$. Then $A$ defines a fit or “fuzzy unit” vector of length $n$: $A = (a(x_1), \ldots, a(x_n)) = (a_1, \ldots, a_n)$ where $a_i = \text{Degree}(x_i \in A)$. Pointwise operations define set operations: $A^C = (1 - a_1, \ldots, 1 - a_n)$, $A \cap B = (\min(a_1,b_1), \ldots, \min(a_n,b_n))$, and $A \cup B = (\max(a_1,b_1), \ldots, \max(a_n,b_n))$. Find $A^C$, $A \cap A^C$, $A \cup A^C$ for each of the following eight sets below.

\[C = (1 \quad 0 \quad 1)\] \[D = (0.4 \quad 0.5 \quad 0.6)\] \[E = (0.3 \quad 0 \quad 0.1)\] \[F = (0.6 \quad 0 \quad 0)\] \[G = (0.3 \quad 0.3 \quad 0.3)\] \[H = (0.5 \quad 0.1 \quad 0.4)\] \[I = (0.5 \quad 0.9 \quad 0.6)\] \[J = (0.5 \quad 0.5 \quad 0.5)\]

Problem 3

Define the count $c(A)$ of a fuzzy set $A$ as $c(A) = \sum_{i} a_i$ if $A$ is discrete and $c(A) = \int_{c}^{d} a(x) dx$ if $A$ is continuous and a subset of the real interval $[c,d]$. Compute the counts of the eight sets in Problem 2.
Define the fuzzy equality or the degree to which two fuzzy sets equal each other as follows: $E(A,B) = \frac{c(A \cap B)}{c(A \cup B)}$. Compute the fuzzy equalities of all 8 sets with one another and list the 64 results in an 8-by-8 matrix.
Define the fuzzy subsethood $S(A,B) = \text{Degree}(A \subset B)$ as the degree to which $A$ equals the set intersection $A \cap B$: $S(A,B) = E(A, A \cap B)$. Hence $S(A,B) = \frac{c(A \cap B)}{c(A)}$. Prove that we can reverse things and write fuzzy equality in terms of fuzzy subsethood. Then compute the subsethoods of all 8 sets above with one another and list the 64 results in an 8-by-8 matrix. Note that if $A$ is binary then $S(X,A) = \frac{n_A}{n}$ is the relative frequency of $n_A$ successes in $n$ trials.
Define the degree of fuzziness or vagueness of a set as the degree to which the set equals its own set complement: $F(A) = E(A, A^C)$. Hence $F(A) = \frac{c(A \cap A^C)}{c(A \cup A^C)}$. Hence $F(A) = 0$ iff $A$ is binary iff $A$ is a cube vertex. Compute the degrees of fuzziness for each of the above 8 sets. Then prove that the three distinct concepts of equality, subsethood, and fuzziness coincide in the following case: \[E(A, A^C) = F(A) = S(A \cup A^C, A \cap A^C).\]

Problem 4

A finite discrete probability density $P = (p_1, \ldots, p_n)$ is just a fuzzy set with a unit count: $c(P) = 1$. So we can compute the fuzziness of $P$ as $F(P)$. Suppose first that every probability element is such that $p_i \le \frac{1}{2}$. Then find $F(P)$. Suppose next that there is one probability element $p_j$ such that $p_j > \frac{1}{2}$. Then find $F(P)$. Use these results to prove that the fuzziness of a probability distribution linearly goes to zero with dimension $F(P) \le \frac{1}{n-1}$.

Problem 5

Consider the continuous fuzzy set $A_1$ on the interval $[0,1]$ with exponential set function $a_1(x) = e^{-2x}$. Find its fuzziness $F(A_1)$. Consider next the constant midpoint set function $A_2$ on the same interval with $a_2 = \frac{1}{2}$. What is its fuzziness? Find the two subsethood values $S(A_1, A_2)$ and $S(A_2, A_1)$.

--- title: "Homework #10" subtitle: "EE 503: Fall 2025" format: html: toc: false number-sections: false code-fold: show code-tools: true theme: cosmo --- ::: {.callout-important} ## Assignment Details **Assigned:** 28 October **Due:** Tuesday, 04 November at 12:00 **BrightSpace Assignment:** Submit your solutions to BrightSpace ::: ::: {.callout-warning} ## Instructions Write your solutions to these homework problems on separate sheets of paper. Submit your work to Brightspace by the due date. Show all work and box answers where appropriate. Do not guess. ::: --- ## Problem 1 Suppose random variable $X$ has a discrete probability density $P(X) = (\frac{1}{2}, \frac{1}{4}, \frac{1}{8}, \frac{1}{8})$ and random variable $Y$ has density $P(Y) = (\frac{1}{4}, \frac{1}{4}, \frac{1}{4}, \frac{1}{4})$ and both have joint density $P(X,Y)$ where the element in the $i$th row and $j$th column is $p(x_i, y_j)$: $$P(X, Y) = \begin{pmatrix} \frac{1}{8} & \frac{1}{16} & \frac{1}{16} & \frac{1}{4} \\ \frac{1}{16} & \frac{1}{8} & \frac{1}{16} & 0 \\ \frac{1}{32} & \frac{1}{32} & \frac{1}{16} & 0 \\ \frac{1}{32} & \frac{1}{32} & \frac{1}{16} & 0 \end{pmatrix}.$$ Define the entropy of $X$ in bits as $H(X) = -\sum_{i=1}^{n} p_i \log_2 p_i$, the conditional entropy $H(X|Y) = -\sum_{i=1}^{n} \sum_{j=1}^{n} p(x_i, y_j) \log_2 p(x_i | y_j)$, and the joint entropy as $H(X,Y) = -\sum_{i=1}^{n} \sum_{j=1}^{n} p(x_i, y_j) \log_2 p(x_i, y_j)$. Define the mutual information $I(X,Y) = \sum_{i=1}^{n} \sum_{j=1}^{n} p(x_i, y_j) \log_2 \frac{p(x_i,y_j)}{p(x_i) p(y_j)}$. Then $I(X,Y) \ge 0$ holds. First compute $H(X)$, $H(Y)$, $H(X,Y)$, $H(X|Y)$, $H(Y|X)$, and $I(X,Y)$ for the above densities. Then prove that $H(X,Y) = H(X) + H(Y|X)$ and $I(X,Y) = H(X) - H(X|Y)$ and verify for the above densities. Thus $I(X,Y) = H(X) + H(Y) - H(X,Y)$. Verify this too. ## Problem 2 A subset $A \subset X$ is fuzzy iff it has a multivalues indicator function $I_A: X \rightarrow [0,1]$ or more simply $a: X \rightarrow [0,1]$. So a finite fuzzy set $A$ defines a point in the $n$-dimensional unit hypercube: $A \subset [0,1]^n$. Then $A$ defines a *fit* or "fuzzy unit" vector of length $n$: $A = (a(x_1), \ldots, a(x_n)) = (a_1, \ldots, a_n)$ where $a_i = \text{Degree}(x_i \in A)$. Pointwise operations define set operations: $A^C = (1 - a_1, \ldots, 1 - a_n)$, $A \cap B = (\min(a_1,b_1), \ldots, \min(a_n,b_n))$, and $A \cup B = (\max(a_1,b_1), \ldots, \max(a_n,b_n))$. Find $A^C$, $A \cap A^C$, $A \cup A^C$ for each of the following eight sets below. $$C = (1 \quad 0 \quad 1)$$ $$D = (0.4 \quad 0.5 \quad 0.6)$$ $$E = (0.3 \quad 0 \quad 0.1)$$ $$F = (0.6 \quad 0 \quad 0)$$ $$G = (0.3 \quad 0.3 \quad 0.3)$$ $$H = (0.5 \quad 0.1 \quad 0.4)$$ $$I = (0.5 \quad 0.9 \quad 0.6)$$ $$J = (0.5 \quad 0.5 \quad 0.5)$$ ## Problem 3 a. Define the count $c(A)$ of a fuzzy set $A$ as $c(A) = \sum_{i} a_i$ if $A$ is discrete and $c(A) = \int_{c}^{d} a(x) dx$ if $A$ is continuous and a subset of the real interval $[c,d]$. Compute the counts of the eight sets in Problem 2. b. Define the fuzzy equality or the degree to which two fuzzy sets equal each other as follows: $E(A,B) = \frac{c(A \cap B)}{c(A \cup B)}$. Compute the fuzzy equalities of all 8 sets with one another and list the 64 results in an 8-by-8 matrix. c. Define the fuzzy subsethood $S(A,B) = \text{Degree}(A \subset B)$ as the degree to which $A$ equals the set intersection $A \cap B$: $S(A,B) = E(A, A \cap B)$. Hence $S(A,B) = \frac{c(A \cap B)}{c(A)}$. Prove that we can reverse things and write fuzzy equality in terms of fuzzy subsethood. Then compute the subsethoods of all 8 sets above with one another and list the 64 results in an 8-by-8 matrix. Note that if $A$ is binary then $S(X,A) = \frac{n_A}{n}$ is the relative frequency of $n_A$ successes in $n$ trials. d. Define the degree of fuzziness or vagueness of a set as the degree to which the set equals its own set complement: $F(A) = E(A, A^C)$. Hence $F(A) = \frac{c(A \cap A^C)}{c(A \cup A^C)}$. Hence $F(A) = 0$ iff $A$ is binary iff $A$ is a cube vertex. Compute the degrees of fuzziness for each of the above 8 sets. Then prove that the three distinct concepts of equality, subsethood, and fuzziness coincide in the following case: $$E(A, A^C) = F(A) = S(A \cup A^C, A \cap A^C).$$ ## Problem 4 A finite discrete probability density $P = (p_1, \ldots, p_n)$ is just a fuzzy set with a unit count: $c(P) = 1$. So we can compute the fuzziness of $P$ as $F(P)$. Suppose first that every probability element is such that $p_i \le \frac{1}{2}$. Then find $F(P)$. Suppose next that there is one probability element $p_j$ such that $p_j > \frac{1}{2}$. Then find $F(P)$. Use these results to prove that the fuzziness of a probability distribution linearly goes to zero with dimension $F(P) \le \frac{1}{n-1}$. ## Problem 5 Consider the continuous fuzzy set $A_1$ on the interval $[0,1]$ with exponential set function $a_1(x) = e^{-2x}$. Find its fuzziness $F(A_1)$. Consider next the constant midpoint set function $A_2$ on the same interval with $a_2 = \frac{1}{2}$. What is its fuzziness? Find the two subsethood values $S(A_1, A_2)$ and $S(A_2, A_1)$.