Random Walk Problem

Notice: This article is just a short discussion on Random Walk problem, I compute \(E(X^2)\) in this article. After I read some materials, from a programmer's perspective, I have found that this problem is not just simple as I think, and it's not easy to compute \(E(\lvert X \rvert)\).

For more details, please read:

• https://mathworld.wolfram.com/RandomWalk1-Dimensional.html
• https://mathworld.wolfram.com/RandomWalk2-Dimensional.html

What is a Random Walk problem?

Refer to: https://en.wikipedia.org/wiki/Random_walk

In mathematics, a random walk is a random process that describes a path that consists of a succession of random steps on some mathematical space.

And we want to know the "distance" (value of mathematical expectation) between the object and the origin position after \(n\) steps.

1D Random Walk

The inferences of expectation formula:

• E(aX+bY) = E(aX) + E(bY) holds even if X, Y are not independent.
• E(XY) = E(X) * E(Y) holds if and only if X, Y are independent.

An elementary example of a random walk is the random walk on the integer number line \(\mathbb{Z}\) , which starts at 0 and at each step moves +1 or -1 with equal probability.

• To define this walk formally, take independent random variables \({ Z_{1},Z_{2},\dots }\), where each variable is either 1 or −1, with a 50% probability for either value.

• Set ${ S_{0}=0} $ and \({ S_{n}=\sum _{j=1}^{n}Z_{j}}\).

• The series \({ \{S_{n}\}}\) is called the simple 1D-random walk on \(\mathbb{Z}\).

• This series (the sum of the sequence of −1 and 1) gives the net distance walked, if each part of the walk is of length one. Obviously, the expectation \(E(S_{n})\) is zero. That is

  \[E(S_{n})=\sum _{j=1}^{n}E(Z_{j})=0 \]

Using the independence of the random variables and the fact that \(E(Z_{i}^{2})=1\), shows that:

\[E(S_{n}^{2})=\sum _{i=1}^{n}E(Z_{i}^{2})+2\sum _{1\leq i<j\leq n}E(Z_{i}Z_{j})=n \]

This hints that \(E(|S_{n}|)\), the expected absolute distance after \(n\) steps, should be \(O(\sqrt{n})\).

Actually,

\[\lim_{n \to \infty} \frac{E(\lvert S_n \rvert)}{\sqrt{n}} = \sqrt{\frac{2}{\pi}} \]

That is, when \(n\) gets large enough, \(E(\lvert S_n \rvert)\) is close to \(\sqrt{2n / \pi}\) .

We should remember this inference: in 1D random walk, the expectation of square of Euler distance is:

\[E(\text{dis}^2) = n \]

where \(n\) denote the object have walked \(n\) steps.

This conclusion will be used in the comming section "2D Random Walk".

2D Random Walk

• Suppose there is a 2D plane, and there is a robot at the original position (0, 0).
• Each step, the robot can walk ONE unit toward one of the 4 directions.
• Find the expectation of the distance (Euler distance) from the robot's position to the origin (0, 0), after n steps.

Math Solution

The movements on X-axis and Y-axis are independent, i.e. the robot can move n / 2 steps along both X-axis and Y-axix, respectively.

\[E(\text{dis}^2) = E(X^2) + E(Y^2) = \frac{n}{2} + \frac{n}{2} = n \]

DP Solution

We can solve this problme via dynamic programming.

Suppose given n, then the robot can mostly reach positions in a grid with size (2n + 1) * (2n + 1).

Let N = 2 * n + 1, and suppose the original position of robot is (N/2, N/2) = (n, n).

Let dp[k, i, j] denote the number of possible paths to reach (i, j) in k seconds (i.e. the robot is allowed to walk k steps).

Then we can have the state equation:

\[dp[k, i, j] = \sum_{}{dp[k-1, i + dx, j + dy]} \]

where dx, dy denote the 4 directions around (i, j).

The initial conditions is:

dp[0, N/2, N/2] = 1

How can we get the expectation distance after n steps?

\[\text{E}_n = \frac{\sum_{i, j}({dp[n, i, j] \cdot \text{distance}(i, j)})}{\sum_{i, j}{dp[n, i, j]}} \]

where distance(i, j) denote the distance from (i, j) to original position (N/2, N/2).

By the way, the probability of reaching position (i, j) is

\[P_{(i, j)} = \frac{dp[n, i, j]}{\sum_{i, j}dp[n, i, j]} \]

Here is the source code.

using vec = vector<int>;
using vec2 = vector<vec>;
using vec3 = vector<vec2>;
using pair_t = pair<int, int>;
/* The function to define distance from (x0, y0) to (x1, y1),
 * here we define it as Euler distance (or its square value).
 */
double dis(int x0, int y0, int x1, int y1)
{
    // return sqrt(pow(abs(x0 - x1), 2) + pow(abs(y0 - y1), 2));
    return pow(abs(x0 - x1), 2) + pow(abs(y0 - y1), 2);
}

bool valid(int idx, int limit)
{
    return 0 <= idx && idx < limit;
}
uint64_t getdp(vec3 &dp, int k, int i, int j)
{
    int n = dp.size(), N = dp[0].size();
    return valid(k, n) && valid(i, N) && valid(j, N) ? dp[k][i][j] : 0;
}

/* Return the expectation E(dis^2). */
double RandomWalk2D(int n)
{
    // If the object can be walk toward 8 directions, use this line of code
    // vector<pair_t> dirs = {{-1, -1}, {-1, 0}, {-1, 1}, {0, -1}, {0, 1}, {1, -1}, {1, 0}, {1, 1}};
    vector<pair_t> dirs = {{-1, 0}, {1, 0}, {0, 1}, {0, -1}};
    int N = 2 * n + 1;
    int x0 = n, y0 = n;
    vec3 dp(n + 1, vec2(N, vec(N, 0)));
    dp[0][x0][y0] = 1;

    for (int k = 1; k <= n; ++k)
        for (int i = 0; i < N; ++i)
            for (int j = 0; j < N; ++j)
                for (auto [dx, dy] : dirs)
                    dp[k][i][j] += getdp(dp, k - 1, i + dx, j + dy);

    double total = 0, exp = 0;
    for (int i = 0; i < N; ++i)
    {
        for (int j = 0; j < N; ++j)
        {
            total += dp[n][i][j];
            exp += dp[n][i][j] * dis(x0, y0, i, j);
        }
    }
    return exp / total;
}

int main()
{
    for (int n = 1; n <= 10; ++n)
    {
        double res = RandomWalk2D(n);
        cout << res << '\n';
        assert(abs(res - n) < 1e-6);  // the "distance" is defined as square Euler distance
    }
}

Follow Up

If in each time, the robot is allowed to walk one unit toward one of the 8 directions (shown as follows), what is the expectation \(E(\text{dis}^2)\) ?

X X X
X O X
X X X

Math Solution

Let \(X, Y\) denote the movement on X-axis and Y-axis, respectively, And \(Z\) denote the movement along two diagonals.

Obviously, they are independent. Then we have

\[E(\text{dis}^2) = E(X^2) + E(Y^2) + E(Z^2) = \frac{2}{8}n + \frac{2}{8}n + \frac{4}{8} \cdot 2n = \frac{3}{2}n \]

DP Solution

To verify the formula here, we can make some modifications on above code.

Let dirs become:

vector<pair_t> dirs = {
    {-1, -1}, {-1, 0}, {-1, 1}, 
    {0, -1}, {0, 1}, 
    {1, -1}, {1, 0}, {1, 1}
};

For n in range of [1, 10], it will output:

1 1.5
2 3
3 4.5
4 6
5 7.5
6 9
7 10.5
8 12
9 13.5
10 15

Similar leetcode problem: 688. Knight Probability in Chessboard