Iterative algorithm(meta-algo I)

abstract: value simplicity

abstract away the inessential features of a problem

goal: understand and think about complex algorithm in simple ways.

If you do not get all the details of the new algorithm correct, it won't be a disaster.

If you do not get the steps of the "meta algorithm" correct, it will be a disaster.

-Meta Math

Asymptotic Growth

To think about algorithms in such way that their correctness is transparent.

loop ( until done)
     take step

end loop

a good way to structure many problems:

store the key information you currently know in some data structure.

In the main loop, take a step forward towards destination.

By making a simple change to this data.

The current "state" of computation is determined by values fo all variables.

Do now worry about the entire computation.

Take one step at a time.

By induction,

1    S(0)


2    for any S(i) ==> S(i+1)

==>

3    S(n)...

 

1. Define Problem
2. Define Loop invariants
3. Define Measure of Progress
4. Define Step
5. Define Exit Condition
6. Maintain Loop Inv
7. Make progress
8. Initial Conditions
9. Ending

max(a,b,c)
"""preCond: Input has 3 numbers."""

    m=a
    """assert: m is max in {a}"""
    if(b>m)
        m=b;
    endif
    """assert: m is max in {a,b}"""
    if(c>m)
        m=c
    endif
    """assert: m is max in {a,b,c}"""
    return (m);

"""postCond: return max in {a,b,c}"""

Loop-Inv: m is max in A[1..i]

Correctness:

<PreCond> + <Code> ===    <PostCond>

<assert 1> +<one step> === <assert 2>

Before you start coding, what do you want be true in the middle of your computation?

You need to know,and also let your reader know.

 

Insert sort loop invariant

• some subset of elements are sorted
• the remaining elements are off to the side

select arbitrary element from side.

insert it where it belongs.

making progress while maintaining the loop invariant.

Runnint time: insert an element into a list of size i take O(i) time.

total = 1+2+3+...+n = O(n²)

Your job is only to:

1. trust who passes you the baton,
2. go around once,
3. pass on the baton

HOPE YOU ARE BEGINNING TO APPRECIATE loop invariants TO DESCRIBING ALGORITHMS.

 explaining insertion sort

We maintain a subset of elements sorted within a list. The remaining elements are off to the side somewhere. Initially, think of the first element in the array as a sorted list of length one. One at a time, we take one of the elements that is off to the side and we insert it into the sorted list where it belongs. This gives a sorted list that is one element longer than it was before. When the last element has been inserted, the array is completely sorted.

 explaining selection sort

We maintain that the k smallest of the elements are sorted in a list. The larger elements are in a set on the side. Initially, with k=0, all the elements are in the set. Progress is made by finding the smallest element in the remaining set of large elements and adding this selected elements at the end of the sorted list of elements. This increases k by one. Stop with k=n. At this point, all the elements have been selected and the list is sorted.

Coming up with loop invariant is the hardest part of designing an algorithm

Don't start coding,

You must design a working algorithm first.

Describe or draw a picture of what the data structure might look like after a number of these steps.

Picture from the middle.

Leap into the middle of the algorithm.

Loop Invariants are pictures of current state. No actions. Not algorithms.

 

Typical types of loop invariants

more of the input

more of the output

narrowed the search space(eg. binary search tree)

 

binary search : define loop invariant

• maintain a sub-list
• if the key is contained in the original list, then the key is contained in the sub-list.

binary search : define step

• cut sub-list in half
• determine which half key would be in
• keep that half

binary search: initial condition

• the sub-list is the entire original list.
• if the key is contained in the original list, then the key is contained in the sub-list.

binary search: ending algorithm

• if the key is contained in the original list, then the key is contained in the sub-list.
• sub-list contains one element.
• loop invariant keep true, even if the key is not in the list.

Running time: 

the sub-list is of size n, n/2,n/4,...1

each step O(1) time.

total : O(logⁿ)

 

 Running Time:

Each iteration takes O(1) time, Total =O(n)

so, balanced partition, O(nlogⁿ).

 

Time complexity of addition

T(n) = the amount of time addition uses to add two n-bits numbers = O(n) = linear time.