mx's blog - algorithm

codejam常用(python)解题工具

2016-05-27T18:00:00+02:00

总结一下用python撸codejam时常用的一些库, 并且给一些简单的例子. 发现用python撸codejam非常合适: codejam的时间要求不严格(4/8分钟), 而且程序只要本地运行. 正好可以使用python简洁的语法和丰富的函数库.

collections

py自带的一些好用的数据结构...
https://docs.python.org/2/library/collections.html

from collections import Counter, deque, defaultdict

itertools

主要是用来穷举的时候它里面一些函数很好用...

https://docs.python.org/2/library/itertools.html

>>> from itertools import product, combinations   
>>> a = 'ABCD'; b='EFG'   
>>> for p in product(a,b):    
print p   
...        
('A', 'E')   
('A', 'F')   
('A', 'G')   
('B', 'E')   
('B', 'F')   
('B', 'G')   
('C', 'E')   
('C', 'F')   
('C', 'G')   
('D', 'E')   
('D', 'F')   
('D', 'G')   
>>> for c in combinations(a, 2): print c    
...        
('A', 'B')   
('A', 'C')   
('A', 'D')   
('B', 'C')   
('B', 'D')   
('C', 'D')   
>>> for p in permutations(b,2): print p   
...    
('E', 'F')   
('E', 'G')   
('F', 'E')   
('F', 'G')   
('G', 'E')   
('G', 'F')

bitmap

聪明一点的穷举需要用bitmap... 实测可以加速十倍...

use bitmap for combinations (2^N possibilities)

(N elements, each element 2 choices)
for mask in xrange(1<<N): ...

set/clean Kth bit

set: bm |= 1<<k

clean: bm &= ~(1<<k)

count nb of 1s in a bitmap

bin(bm).count('1')

networkx

常用的图论算法都在里面了. nx最棒的是任何hashable的object都可以用来作为节点的index, 再想想用C++的bgl, 简直蛋疼... https://networkx.readthedocs.io/en/stable/

constructing graph

>>> import networkx as nx   
>>> G = nx.DiGraph() # use `Graph` for undired graph, `MultiGraph` for dup-edges   
>>> G.add_node('a') # any hashable obj can be used as node index   
>>> G.add_edge(1,2) # missing nodes will be automatically added   
>>> G.add_edge(1,3) # if G is undired(`Graph`), 1-->3 and 3-->1 will be added   
>>> G.nodes()   
['a', 1, 2, 3]   
>>> G.edges()   
[(1, 2), (1, 3)]   
>>> G.add_edge(1,2); G.add_node('a') # nx ignores duplicate adding edges/nodes    
>>> G.nodes()   
['a', 1, 2, 3]   
>>> G.edges()   
[(1, 2), (1, 3)]   
>>> G[1] # outgoing edges from a node   
{2: {}, 3: {}}   
>>> G[1][2]['color']='blue' # easily add edge properties    
>>> G[1]   
{2: {'color': 'blue'}, 3: {}}   
>>> G.add_edge(1,2, capacity=1) # this is another way to add property   
>>> G.edge   
{'a': {}, 1: {2: {'color': 'blue', 'capacity': 1}, 3: {}}, 2: {}, 3: {}}   
>>> G.node['a']['cat']='string node' # can also be: G.add_node('a', cat='string node')   
>>> G.node   
{'a': {'cat': 'string node'}, 1: {}, 2: {}, 3: {}}

DiGraph: topo-sort, cycle-detection, strongly connected component

http://networkx.readthedocs.io/en/stable/reference/algorithms.shortest_paths.html

>>> import networkx as nx    
>>> G = nx.DiGraph()   
>>> G.add_edge(1,2); G.add_edge(1,3); G.add_edge('a','b')   
>>> list( nx.strongly_connected_components(G) )   
[set(['b']), set(['a']), set([2]), set([3]), set([1])]   
>>> nx.topological_sort(G)   
[1, 2, 3, 'a', 'b']   
>>> G.add_edge(2,3); G.add_edge(3,1); G.add_edge('b','a')   
>>> list( nx.simple_cycles(G) )   
[[1, 3], [1, 2, 3], ['a', 'b']]   
>>> list( nx.strongly_connected_components(G) )   
[set(['a', 'b']), set([1, 2, 3])]   
>>> G.add_edge(3,4); G.add_edge(4,'a')   
>>> nx.shortest_path(G,1,'a')   
[1, 3, 4, 'a']   
>>> G.add_edge(1,3,weight=2); G.add_edge(1,2,weight=3)   
>>> nx.shortest_path(G,1,'a')   
[1, 3, 4, 'a']   
>>> nx.shortest_path_length(G,1,'a')   
3   
>>> nx.shortest_path_length(G,1,'a','weight') # set attribut edge 'weight' as weight, (if not present, weight=1 )   
4

Undirected Graph: connected component, MST

http://networkx.readthedocs.io/en/networkx-1.11/reference/generated/networkx.algorithms.mst.minimum_spanning_tree.html#networkx.algorithms.mst.minimum_spanning_tree

>>> G = nx.Graph()   
>>> G.add_edge(1,2); G.add_edge(1,3); G.add_edge('a','b')   
>>> list( nx.connected_components(G) )   
[set(['a', 'b']), set([1, 2, 3])]   
>>> G.add_edge(2,3)   
>>> mst =  nx.minimum_spanning_tree(G) # returns a new graph   
>>> mst.edges()   
[('a', 'b'), (1, 2), (1, 3)]   
>>> G.add_edge(1,3,weight=2) # mst takes attribut 'weight', if no present, weight=1   
>>> nx.minimum_spanning_tree(G).edges()   
[('a', 'b'), (1, 2), (2, 3)]

maxflow

http://networkx.readthedocs.io/en/networkx-1.11/reference/algorithms.flow.html

>>> import networkx as nx   
>>> G = nx.DiGraph()   
>>> G.add_edge('x','a', capacity=3.0)   
>>> G.add_edge('x','b', capacity=1.0)   
>>> G.add_edge('a','c', capacity=3.0)   
>>> G.add_edge('b','c', capacity=5.0)   
>>> G.add_edge('b','d', capacity=4.0)   
>>> G.add_edge('d','e', capacity=2.0)   
>>> G.add_edge('c','y', capacity=2.0)   
>>> G.add_edge('e','y', capacity=3.0)   
>>> flow_value, flow_dict = nx.maximum_flow(G, 'x', 'y')   
>>> flow_value   
3.0   
>>> print(flow_dict['x']['b'])   
1.0

maximum matching

NB: maximum matching != maximal matching...
there are maximum-matching functions for general undir graph (max_weight_matching) and for bipartitie graph (maximum_matching), the one for bipartite graph is faster, the general one takes O(V**3).

http://networkx.readthedocs.io/en/stable/reference/generated/networkx.algorithms.matching.max_weight_matching.html?highlight=maximum_matching

>>> G = nx.Graph()   
>>> G.add_edges_from([(1,2),(2,3),(3,4),(4,5)])   
>>> mate = nx.max_weight_matching(G, maxcardinality=True)#mate[v] == w if node v   
 is matched to node w.   
>>> mate   
{2: 3, 3: 2, 4: 5, 5: 4}   
>>> nx.is_bipartite(G)   
True   
>>> mate=nx.bipartite.maximum_matching(G)   
>>> mate   
{1: 2, 2: 1, 3: 4, 4: 3}

and there are vertex cover algorithms as well......

pulp

线性规划的库, 供了非常好用的接口来构造LP问题, 增加约束或者定义objective只要用prob+=[expression]就好了, 基本上看看例子就能上手. 面对选择问题的时候线性规划是不错的方法 -- 如果计算速度可以足够快的话...

https://pythonhosted.org/PuLP/pulp.html

>>> from pulp import *   
>>> x = LpVariable("x", 0, 3)   
>>> y = LpVariable("y", 0, 1, 'Integer') # var category can be integer   
>>> prob = LpProblem("myProblem", LpMinimize)   
>>> prob += x + y <= 2 # add constraint   
>>> prob += -4*x + y # add objective   
>>> status = prob.solve() # solve using default solver   
>>> status = prob.solve(GLPK(msg = 0)) # or use glpk solver   
>>> LpStatus[status]   
'Optimal'   
>>> value(prob.objective) # see objective value   
-8.0   
>>> value(x) # see variable value    
2.0

关于nx和pulp的应用可以参考上篇文章.

codejam2015-round2-pbC 的三种解法

2016-05-27T00:00:00+02:00

昨天做的一道codejam题目, 这个题目的三种解法都非常有代表性, 特此一记.

题目链接在这里:
https://code.google.com/codejam/contest/8234486/dashboard#s=p2

Elliot's parents speak French and English to him at home. He has heard a lot of words, but it isn't always clear to him which word comes from which language! Elliot knows one sentence that he's sure is English and one sentence that he's sure is French, and some other sentences that could be either English or French. If a word appears in an English sentence, it must be a word in English. If a word appears in a French sentence, it must be a word in French.
Considering all the sentences that Elliot has heard, what is the minimum possible number of words that he's heard that must be words in both English and French?

Input
The first line of the input gives the number of test cases, T. T test cases follow. Each starts with a single line containing an integer N. N lines follow, each of which contains a series of space-separated "words". Each "word" is made up only of lowercase characters a-z. The first of those N lines is a "sentence" in English, and the second is a "sentence" in French. The rest could be "sentences" in either English or French. (Note that the "words" and "sentences" are not guaranteed to be valid in any real language.)

Output
For each test case, output one line containing "Case #x: y", where x is the test case number (starting from 1) and y is the minimum number of words that Elliot has heard that must be words in both English and French.

Limits
1 ≤ T ≤ 25.
Each word will contain no more than 10 characters.
The two "known" sentences will contain no more than 1000 words each.
The "unknown" sentences will contain no more than 10 words each.
Small dataset
2 ≤ N ≤ 20.
Large dataset
2 ≤ N ≤ 200.

我的codejam程序模板长这样:

def readval(typ=int):   
    return typ( raw_input() )

def readvals(typ=int):   
    return map( typ, raw_input().split() )

def testcase(cas):   
    print 'Case #%d: %d' % ( cas, res )

if __name__=='__main__':   
    T = int(raw_input())   
    for i in xrange(T):   
        testcase(i+1)

法1: 穷举 bruteforce (for small testcase)

在small set(N=20) 中, 对于那18个未知语言的句子, 每句可能是英语或法语. 那么穷举所有可能性, 然后选择双语单词最少的即可. 2^18约等于几十万 , 按说python还是可以handle的 (2^10=1k, 2^ 20=1M, python每秒大约能循环几百万次).

naive bruteforce

生成所有的可能性组合, python的itertools包里提供了combinations函数, combinations(iterable, r)返回所有在iterable中大小为r的子集(的迭代器):

>>> from itertools import combinations   
>>> for c in combinations('ABCD', 2): print c   
('A', 'B')   
('A', 'C')   
('A', 'D')   
('B', 'C')   
('B', 'D')   
('C', 'D')

用combination函数即可实现枚举每句话是英语还是法语的功能.

当每句话是英语还是法语已经确定以后, 可以用集合取交集的方法(set1.intersection(set2))得到两种语言一共有多少个重复的单词. 程序长这样:

def testcase(cas): # not fast enough...   
    N = readval()   
    En = set( readvals(str) )   
    Fr = set( readvals(str) )   
    if N==2:    
        print 'Case #%d: %d' % ( cas, len(En.intersection(Fr)) )   
        return   
    scent = []   
    for i in xrange(N-2):   
        scent.append( set(readvals(str)) )   
    def partition(engsubset):    
        allEn, allFr = En.copy(), Fr.copy()   
        for i in xrange(N-2):    
            if i in engsubset: allEn.update(scent[i])   
            else: allFr.update(scent[i])   
        return len( allEn.intersection(allFr) )    
    possibleres = []   
    for l in xrange(N-2):    
        for engsubset in combinations(xrange(N-2), l):    
            possibleres.append(partition(engsubset))   
    res = min(possibleres)   
    print 'Case #%d: %d' % ( cas, res )

然而, 下载了small testcase以后运行程序, 速度还是不够快, 大约要十分钟才有结果 — 而codejam的提交时间限制是4分钟啊... 程序每次检查2^18种可能性, 但是由于每次检查都要进行集合的union和intersection操作(这种操作的效率并不高, 甚至C++里也是一样), 这个操作太耗费时间了所以不行...

bruteforce using bitmap

需要更加聪明的穷举方法, 自然的想法就是用bitmap(或者叫bitvector?). 之前的两个基本操作都可以用bitmap完成:

枚举各个句子的语言种类: 如果每个句子用一位来表示的话(1代表英语, 0代表法语), 那么用N位的bitmap即可表示一种情形. 这个bitmap只需从0增加到2^N-1, 就把2^N个可能性都遍历了. (实际上是2^N-2个可能性, 因为前两个句子已经确定语言了).
两种语言的词汇表进行union/intersection: 假设共有K个不同的单词, 那么用一个K位的bitmap即可表示一种语言包含了哪些单词. 然后集合的union和intersection即可表示为OR和AND的逻辑运算.

bit manipulation蛮subtle的, 不过习惯了就好... 另外python的integer可以任意长度, 不用像C/java那样考虑bitmap位数大于64的情况, 还是非常方便的. 代码如下:

def testcase(cas): # use bitmap instead of set to speedup for small case !!    
    N = readval()   
    scent = []; allwords = set()   
    for i in xrange(N):    
        scent.append( readval(str) )   
        allwords.update( scent[i].split() )   
    words = sorted(allwords)    
    # if K distinct words in total, each sentence can be reprensented as a K-bit bitmap   
    bitmaps = []    
    for i in  xrange(N): #construct bitmaps   
        bm = 0   
        for wd in scent[i].split():   
            bm |= ( 1<< words.index(wd) )   
        bitmaps.append(bm)   
    res = 1e10   
    # look for all combinations    
    for i in xrange(1<<(N-2)):   
        en = bitmaps[0]; fr = bitmaps[1]   
        for k in xrange(N-2):   
            if (1<<k) & i > 0: en |= bitmaps[k+2]   
            else: fr |= bitmaps[k+2]   
        res = min( res, bin(en&fr).count('1') )   
    print 'Case #%d: %d' % ( cas, res )

以上代码在我电脑上执行small的时候大约花费1分钟, 比最开始用set的速度提高了十倍. 当然, 对于large的case这种解法肯定就超时了...

法2: 最大流 maxflow

这个方法也是官方analysis里提供的答案. 如果将所有句子S以及所有单词w看作节点, 每个句子的节点S于它包含的单词的节点w相连, 问题转化成了从节点S1到节点S2的vertex cut问题 (选取最小的节点集合, 将该集合的节点去掉以后S1和S2不再联通).

然后这个问题又可以转为边的min cut问题: 只需要把每个单词节点w分成左右两个节点w1和w2, 并且这样添加边(假设w在句子S中):

w1-->w2, capacity=1
S-->w1, capacity=INF
w2-->S, capacity=INF

在这样构造的图里计算maxflow即可得到mincut, 也就是题目的答案...

在实现这个解法的时候用到了networkx这个包, 它提供了比bgl好用100倍的接口(虽然也比bgl慢差不多100倍 ==...). 任何hashable的object都可以用来作为节点的index, 所以写起来非常舒服, 15行搞定:

def testcase_maxflow(ind): # using maxflow !   
    INF = 1e10   
    import networkx as nx   
    G = nx.DiGraph()   
    N = readval()   
    for i in xrange(N):   
        words = readvals(str)   
        si = 'sent-%d'%i   
        G.add_node( si )   
        for wd in words:    
            G.add_edge( wd+'_l', wd+'_r', capacity=1 )   
            G.add_edge( si, wd+'_l', capacity=INF )   
            G.add_edge( wd+'_r', si, capacity=INF )   
    flow_value, flow_dict = nx.maximum_flow(G, 'sent-0', 'sent-1')   
    print 'Case #%d: %d' % ( ind, int(flow_value) )

用这个代码即可轻松通过large case... maxflow的建模还真是艺术...orz

法3: 线性规划 (integer) linear programming

写完上面那个解法以后, 我又看了看大神的解法, 发现他居然用的是pulp(python线性规划的包), 于是自己想了一下, 这个问题确实可以用线性规划来建模 !

首先说一句pulp这个包, 它提供了非常好用的接口来构造LP问题, 增加约束或者定义objective只要用prob+=[expression]就好了, 可以说比AMPL好用不少... 下面是一个简单的例子:

>>> from pulp import *   
>>> x = LpVariable("x", 0, 3)   
>>> y = LpVariable("y", 0, 1, 'Integer') # var category can be integer   
>>> prob = LpProblem("myProblem", LpMinimize)   
>>> prob += x + y <= 2 # add constraint   
>>> prob += -4*x + y # add objective   
>>> status = prob.solve() # solve using default solver   
>>> status = prob.solve(GLPK(msg = 0)) # or use glpk solver   
>>> LpStatus[status]   
'Optimal'   
>>> value(prob.objective) # see objective value   
-8.0   
>>> value(x) # see variable value    
2.0

OK, 本题的建模过程如下:
对于每一个句子S, 定义binary的LP变量Se, Se=1表示句子S为英语, =0表示为法语.
对于每个单词w, 定义两个binary变量we, wf, 表示单词w是英(法)语单词.
然后, 再定义变量wef, 它表示单词w既是英语单词又是法语单词. 所以有这个逻辑关系: wef = we AND wf 那么这种逻辑关系如何用线性约束描述呢? 可以这样: wef >= we+wf-1 非常巧妙吧...
(这篇文章总结了各种逻辑关系用线性规划的描述方式, 写的得非常详细. )

要最小化的目标函数就是 sum(wef) 了, 约束除了刚才那个wef >= we+wf-1以外, 还要表达单词和句子之间的关系: 如果一个句子为英(法)语, 那么句子里的每一个单词都为英(法)语. 这是一个se==>we的逻辑关系, 用线性约束表达为: we>=se.

所以整个模型是:

Minimize sum(wef)   
st:   
we >= se   
wf >= (1-se)   
wef >= we+wf-1   
Se[0]==1, Se[1]==0

用pulp编写的代码如下:

def testcase(ind):# formulate it as linear programming    
    N = readval()   
    sentc = []   
    allwords = set()   
    for i in xrange(N):   
        sentc.append( set(readvals(str)) )   
        allwords.update( sentc[-1] )   
    words = sorted( allwords )   
    M = len(words)   
    wordsindex = {words[i]:i for i in xrange(M)} # mapping a word to its index   
    pb = LpProblem('Bilingual', LpMinimize)   
    # LP variables   
    Se = [ LpVariable('Se_'+str(i), cat='Binary') for i in xrange(N) ] # Se[i] = indicator(scentence i is english)   
    we = [ LpVariable('we_'+str(j), cat='Binary') for j in xrange(M) ] # we[j] = indicator(word j is english)    
    wf = [ LpVariable('wf_'+str(j), cat='Binary') for j in xrange(M) ] # wf[j] = indicator(word j is french)    
    wef = [ LpVariable('wef_'+str(j), cat='Binary') for j in xrange(M) ] # wef[j] = indicator(word j is BOTH en and fr)    
    pb += sum( wef )   
    pb += Se[0]==1   
    pb += Se[1]==0   
    for i in xrange(N):    
        si = sentc[i]   
        for wd in si:    
            j = wordsindex[wd]   
            pb += we[j] >= Se[i]   
            pb += wf[j] >= (1-Se[i])   
    for j in xrange(M):    
        pb += wef[j] >= we[j]+wf[j]-1 # # wef[i] = we[i] && wf[i]   
    #~ pb.solve( GLPK(msg=0) )   
    pb.solve(  )   
    res = int( value(pb.objective) )   
    print 'Case #%d: %d' % ( ind, res )

以上代码在small上运行约1分钟, large约3分钟.

另外我发现另一个大神代码里没有用Binary/Integer的lp变量, 也就是说他用的是连续的线性规划! 当我把变量的定义改成:

Se = [ LpVariable('Se_'+str(i), 0, 1) for i in xrange(N) ] # Se[i] = indicator(scentence i is english)   
we = [ LpVariable('we_'+str(j), 0) for j in xrange(M) ] # we[j] = indicator(word j is english)    
wf = [ LpVariable('wf_'+str(j), 0) for j in xrange(M) ] # wf[j] = indicator(word j is french)    
wef = [ LpVariable('wef_'+str(j), 0) for j in xrange(M) ] # wef[j] = indicator(word j is BOTH en and fr)

这样以后, 得到的结果还是正确的!! (不过代码运行时间没有显著的提高).

所以是这个模型中的矩阵满足totally unimodular性质?
不过这个模型似乎并不满足wikipedia里写的那个充分条件?...@@
这里有一篇真•大神的文章是讲如何识别TU矩阵的 (tl;dr......), 那么是不是各种solver内部已经有了自动判断TU的代码, 所以这两种程序的运行时间差不多?...

总而言之这道题目还是非常有意思的, 三种解法都很有代表性, 这里的技巧估计在codejam里会经常用到...

[Algorithms II] Week 6-3 Intractability

2016-02-23T00:00:00+01:00

1. Introduction to Intractability

recall model of computation: DFA
a univeral model of computation: turing machine
→ no more powerful model of computation.
Turing machine can compute any function that can be computed by a physically harnessable process of the natural world.

bottom line: turing machine is a simple and universal model of computation.

Q. which algos are useful in practice?

useful in practice = polynomial time for all inputs

def. a pb is intractable if it cannot be solved in polynomial time.

2 pbs that can be proved to require exp time:

Given a constant-size programme, does it halt in <=K steps ?
Given a N*N chess board position, can the first player force a win ?

Bad news: very few pbs can be proved to require exp time...

2. Search Problems

**Four fundamental problems: **

LSLOVE

Given a system of linear equations, find a solution
var: real numbers
→ guassian elimination

Given a system of linear inequaties, find a solution. (not necessarily find the opt)
var: real numbers

ILP

Given a system of linear inequaties, find a 0-1 solution.
var: 0 or 1

SAT

Given a system of boolean equations, find a binary solution.

Which ones of the 4 foundamental pbs have poly-time solutions?

LSLOVE: Gaussian elimination works in O(n3)
LP: Ellipsoid works in poly-time (simplex also poly-time in practice..)
ILP, SAT: No poly-time algorithm known (or believed to exist) !

All 4 pbs are examples of search problems.

Search pb: given an instance I, find a solution S / report there's no solution.
requirement: able to efficiently (poly-time) check that S is a solution. (that's the case for the above 4 fundamental pbs)

another example:
FACTOR: given a n-bit integer, find a nontrival factor.
(given a solution, simply need to long-divide to check...)

3. P vs. NP

def. NP is the class of all search pbs. (ie. solution be checked efficiently)
NB: classical definition limits to yes-no pbs...
Significance: NP pbs are what scientists and engineers aspire to compute feasibly.
examples:

def. P is the class of search pbs that are solvable in poly-time.

(What scientists and engineers do compute feasibly.)

examples:

Nondeterminism
Nondeterminism machine can guess the solution (donot exist in natural world..). → NFA tries to simulate such a machine...
Ex. int[] a = new int[N];
・ Java: initializes entries to 0 .
・ Nondeterministic machine: initializes entries to the solution!

NP: Search problems solvable in poly time on a nondeterministic Turing machine.

Extended Church-Turing thesis:
P: Search pbs solvable in poly time in natural world.

do we have non-determinism in natural world? ---> natural computers ?
ex. STEINER tree: set of segments connecting given N points.
use soap → doesn't really work...

another example for P/NP: automating creativity
being creative VS appreciating creativity

The central question: does P=NP?
(can you alway avoid brute-force searching and do better?)

Millennium prize by Clay instute.
(among all ways of earning 1M dollars, this might be the most complicated way... @_@...)

4. Classifying Problems

classify pbs like classifying elements into perodic table.

key pb: satisfiablity
SAT. given a sys of boolean eq, find a solution.

exhaustive search: try 2^n possible solutions.
conjecture: no poly-time algo for SAT (ie. intractable)

Assumption: assume the intractability for SAT.
Tool: reduction
def. pb X reduces to pb Y: we can solve pb X with the algo for pb Y.

if SAT poly-reduces to pb Y ⇒ pb Y in (probably) intractable.

SAT poly-reduces to ILP

(all SAT pb can be reduced to 3SAT)

⇒ can be converted to an ILP pb:
for each eq, introduce a var Ci:

5. NP-Completeness

def. an NP pb is NP-complete if all pbs in NP poly-reduces to it.

prop. SAT id NP-complete.
any pb in NP poly-reduces to SAT (reverse direction as last lecture)
pf sketch: convert non-dertiministic turing machine notation to SAT notation...

cor. poly time algo for SAT iff P=NP...

⇒ there pbs are equivalent !

summary:

==...

6. Coping with Intractability

exploit intractability

cryptography ecopoits the hardness of FACTOR pb

Can factor an n-bit integer in n 3 steps on a "quantum computer.”

Coping with intractability

relax one of desired features...

special cases

Develop a heuristic, and hope it produces a good solution.

no guarantee
ex. TSP

Approximation algorithm. Find solution of provably good quality.

Halmiton path

remark: Euler path (each edge once) easy, Halmiton path (each vertex once) NPC...
dfs solution for Halmiton path:

public class Halmiton{   
    private boolean[] marked;   
    private int count=0; // nb of Halmiton paths   
    public Halmiton(Graph G){   
        marked = new boolean[G.V()];   
        for (int v=0; v<G.V(); v++)   
            dfs(G,1,1);    
    }   
    private void dfs(Graph G, int v, int depth){   
        if(depth==G.V()) count++;   
        marked[v]=true;   
        for(int w: G.adj(v))    
            if(marked[w]==false) dfs(G, w, depth+1);   
        marked[v]=flase; // backtrack   
    }   
}

[Algorithms II] Week 6-2 Linear Programming

2016-02-21T17:00:00+01:00

simplex algo: top 10 algo of the 20th century (ever?).

what is linear programming:
a general problem-solving model that works for:
shortest-path, maxflow, MST, matching, assignment, ...

1. Brewer-'s Problem

toy example: choose products to maximize profit.
...
feasible region: a convex polygon.

⇒ optimum solution appears at an extreme point.

standard form of LP

n non-neg variables (j=1..n)
m linear euqations (i=1..m)
input: a_ij, c_j, b_i
output: x_j

to convert inequality to equality (as in the standard form above): add slack var!

def. convex set
for any a and b in set ⇒ 1/2(a+b) is also in set.

extreme point:

def. extreme point
is a point in set that cannot be written as 1/2(a+b) with a b distinct.

extreme point property:
if there exists an potimal solution, then there exists one that is an extreme point.

nb of extreme point is finite
but this nb can be exponential

greedy property:
extreme point is optimal iff no better adj extreme points.

2. Simplex Algorithm

algo. simplex

start at some point

pivot from one extreme point to an adj one (never decrease the obj fcn)

repeat until optimal

We're using the "basis" and "pivoting" to solve LP.

def. basis (基变量) is a subset (size=m) of the n variables.

vars in basis are always non-zero...

basic feasible solution:

set n-m non-basis vars to 0
solve for remaining m vars (with m constraints)
if unique and feasible (matrix invertable)

algo:

initial basic-feasible-solution: start slack vars as basis.
choose a non-basic var as pivot, add it into basis, take some basis var out

ex. pick B as pivot var using constraint 2 (2nd equation):

why picking var B? → its obj coeff is positive
why pivot on 2nd constraint (5A+15B+Sc=480)? →
- RHS > 0 (preserves feasibility)
- minimum ratio rule: min(480/15, 160/4, 1190/20)

stop when no obj-coeff is positive

3. Simplex Implementations

encode standard LP formulation into java 2d array:

public class Simplex{   
    private double[][] a;   
    private int m,n;   
    public Simplex(double[][] A, double[] b, double[] c){   
        m = b.length;   
        n = c.length;   
        a = new double[m+1][n+m+1];   
        for(int i=0; i<m; i++)   
            for(int j=0; j<n; j++)   
                a[i][j] = A[i][j];   
    for(int j=n; j<m+n; j++) a[j-n][j] = 1;      
    for(int j=0; j<m; j++) a[j][n+m] = b[j];    
    for(int j=0; j<n; j++) a[m][j] = c[j];    
    }   
}

simplex algo: just transform initial 2d array into final solution.

choosing pivot variable (find entering column)

Bland's rule. find the first column whose obj-coeff is positive.

private int bland(){   
    for(int q=0; q<m+n; q++)   
        if(a[m][q]>0) return q;   
    return -1;   
}

choosing pivot constraint (find entering row)

minimum ratio rule (if a tie, choose first row).

private int minRatioRule(int q){   
    int p = -1;   
    for(int i=0; i<m; i++){   
        if (a[i][q]<=0) continue;   
        else if (p==-1) p=i;   
        else if (a[i][m+n]/a[i][q] < a[p][m+n]/a[p][q])    
            p=i;   
    }   
    return p;   
}

do the pivot (column q, row p)

like Guassian elimination:

make var q disappear on each row (except for row p);
on row p: make var q's coeff become 1.

public void pivot(int p, int q){   
    for(int i=0; i<m; i++)   
        for(int j=0; j<m+n; j++)   
            if (i!=p && j!=q)   
                a[i][j] -= a[p][j]*a[i][q]/a[p][q];   
    for(int i=0; i<m; i++)    
        if(i!=p) a[i][q] = 0;   
        for(int j=0; j<m+n; j++)   
        if(j!=q) a[p][j] /= a[p][q];   
    a[p][q] = 1;   
}

so the simplex algo is:

public void solve(){   
    while(true){   
        int q = bland();    
        if(q==-1) break; // optimal if -1   
        int p = minRatioRule(q);   
        if(p==-1) break; // unbounded if -1   
        pivot(p,q);   
    }   
}

final solution is just in the array:

remarkable property
in typical applications, simplex terminates after at most 2(m+n) pivots. — whereas nb of extreme points is exp in n !!
ie. LINEAR time in practice!!

other pivot rules:

degeneracy

when choosing new basis, still stay in the same extreme point...

→ might cause cycling
→ bland's rule guarantees finite number of pivots

further improvement:

Best practice. Don't implement it yourself......

(AMPL是个好东西...)

算法的力量:

4. Linear Programming Reductions

reduction to std form (equalities)

Minimization problem: max -1*obj
ineq constraints: add slack var
unbounded var X: replace with X=X0-X1, X0>=0, X1>=0

modeling of LP

identify variables
define constraints
define objective fcn
convert to std form

maxflow by LP

variables: x_uv = flow on edge uv
constraints: capacity, flow conservation
obj: net flow to t

can use LP to solve mincost maxflow easily...

max cardinality bipartite matching by LP

input: bipartite graph
goal: max cardinatlity matching (set of vertex-disjoint edges)

can be reduced to maxflow (见algolab...)

var: x_ij = indicator of person i assigned to job j (0<=x_ij<=1)
constraints: vertex-disjoint
obj: sum of all x_ij

non-trival: cause this is an INTEGER LP...

Th (Von Neumann) (and Poincare?..)
if all RHS=1 ⇒ all extreme points of the polyhedron have integer coord.

and many others...

the profound question: Is there a universal problem-solving model ?
→ P/NP...

"For the time being, the closest thing that we have to universal problem-solving model is LP "

[Algorithms II] Week 6-1 Reductions

2016-02-19T09:00:00+01:00

Goal: classify problems according to computational requirements.
bad new: for huge number of pbs we don't know...

1. Introduction to Reductions

shifing gears:

from individual problems to problem-solving models.
from linear/quard to polynomial/exponential pbs
from implementation details to conceptual framwork

suppose we could (not) solve pb X efficiently
⇒ what else pbs could (not) we solve efficiently ?

def. reduction
Pb X reduces to pb Y if you can use an algo that solves Y to solve X.

for an instance of pb X → transform it into an instance of pb Y → translate the solution for Y to solution for X.

ex1. finding median can reduce to sorting... cost = NlogN+1
ex1. element distinctness can reduce to sorting... cost = NlogN + N

2. Designing Algorithms

algo design: by reduction to problems that we know how to solve (sorting/shortest path/flow/...)

ex1. convex hull reduces to sorting

Gram scan algo... (discussed in algo-I course)
cost = NlogN + N
algo. Gram scan

pick a point with smallest y-coord

sort all points by polar angle wrt the picked point

consider points in this order, discard points that creates clockwise turn

ex2. undirected shortest path (nonneg weights) reduces to directed shortest path

cost: ElogV + E
algo. replace each undir-edge by 2 dir-edge...

3. Establishing Lower Bounds

goal: prove that a pb requires (at least) a certain nb of steps.

ex. any compare-based sorting requires NlogN compares. log(N!) = NlogN

Bad news: very hard to estibalish lower bounds.
Good new: can spread the lower bound NlogN by reducing to sorting (if cost of reduction is small).

def. linear-time reduction
pb X linear-time reduces to pb Y if X can be solved with:
1. linear nb of op for reduction
2. constant nb of calles to Y

ex. almost all reductions we've seen so far...

ex. proof of lower bound for convex hull

prop. sorting linear-time reduces to convex hull
(注意这次是反向的! )
pf.
for an instance of sorting: x1 ... xn
⇒ convert to convex hull instance: (x1, x1^2), ... , (xn, xn^2)

⇒ implication: all (ccw-based) convex hull algo cannot be easier than NlgN ! (otherwise sorting would be easier..)

lesson: Establishing lower bounds through reduction is an important tool in guiding algorithm design efforts.

4. Classifying Problems

prove that pb X and pb Y have the same complexity:

show X linear-time reduces to Y
show Y linear-time reduces to X
conclude that X Y have the same complexity (even if we don't know what it is)

ex. sorting and convex hull...
一个囧囧的脑洞:

ex. integer arithmetic reductions: integer multiplication

integer multiplication: of two N-bit integers.
Its complexity (unknown) is denoted as M(N)
brute force: N^2 ops → so M(N) = Omega(N2)
many other integer ops can reduce to integer multiplication:

what is M(N)?

ex. linear-algebra reductions: matrix multiplication

compute product of 2 N*N matrices.
Its complexity (unknown) is denoted as MM(N)
brute force: N^3
operations that can reduce to matrix-multiplication:

what is MM(N)?

summary

[Algorithms II] Week 5-2 Data Compression

2016-01-04T00:00:00+01:00

1. Introduction to Data Compression

pb: reduce the size of a file, to save space/time for storing/transmitting.
applications: generic file compression(gzip), multimedia (mp3), communication(skype).

From binary data B, ⇒ generate a compressed representation C(B).

lossless compression: get exactly B from C(B)
compression ratio: |C(B)|/|B|(||means number of bits)

Tools

extension to stdio libraries: read/write bits

public class BinaryStdIn{  
    boolean readBoolean(); // read 1 bit and return as a boolean  
    char readChar(); // read 8 bits and return a char  
    char readChar(int r); // read r (<=8) bits and return a char  
    // similar method for int/long/double  
    boolean isEmpty(); // is bitstream empty?  
    void close();  
}


public class BinaryStdOut{  
    void write(boolean b); // write 1 bit  
    void write(char c); // write 8 bits  
    void write(char c, int r); // write r (least-significant) bits of c  
    // similar method for int/long/double  
    void close(); // might add some byte alignment bits   
}

usage example: store a date 12/31/1999

universal date compression?
prop. NO algorithm can compress every bitstring.
pf. by contradiction: repeatedly compress the bitstring ⇒ bit length goes to 0.

2. Run-Length Coding

one simple type of redundancy in bitstream: long runs of repeated bits.
⇒ use 4-bit counts to represent alternating 1s and 0s.

question1: how many bits to represent counts ? ⇒ pick 8 bits (just tradeoff)
question2: what if run length is bigger than max length(2^8)? ⇒ just add length=0 of the alternating bit...

application: JPEG...

code:

public class RunLength{  
    private final static int R = 256; // max run length  
    private final static int lgR = 8;  
    public static void compress(){  
        boolean bit, oldbit = false;  
        int l=0;  
        while(!BinaryStdIn.isEmpty()){  
            bit = BinaryStdIn.readBoolean();  
            if(bit!=old){  
                oldbit = bit;  
                BinaryStdOut.write(l,lgR);  
                l = 0;    
            }  
            else if(l==R-1){  
                BinaryStdOut.write(l,lgR);  
                BinaryStdOut.write(0,lgR);  
                l = 0;  
            }  
            l ++;  
        }  
        BinaryStdOut.write(l,lgR);  
        BinaryStdOut.close();

    }  
    public static void expand(){  
        boolean bit = false;  
        while(!BinaryStdIn.isEmpty()){  
            int l = BinaryStdIn.readInt(lgR);  
            for(int i=0;i<l;i++)  
                BinaryStdOut.write(bit);  
            bit != bit;  
        }  
        BinaryStdOut.close();  
    }  
}

3. Huffman Compression

classique...

idea: variable length code.
ex. Morse code, more freq chars use less chars.
→ pb: ambiguity? one code is the prefix of another...
→ need to use prefix-free code

⇒ use a trie to represent the prefix-free code, in bitstream, use a binary trie:

Huffman Trie Node

define trie nodes having frequences:

public static class Node implements Comparable<Node>{  
    private char ch; // only used for leaves, null for internal nodes  
    private int freq;  
    private final Node left, right; // left-->0, right-->1  
    public Node(char ch, int freq, Node left, Node right){//...}  
    public boolean isLeaf(){  
        return left==null && right==null;}  
    public int compareTo(Node that){  
        return this.freq-that.freq;}  
}

implementation

expansion
in the bitstream, we first put the (serialized) trie in the head.

public void expand{  
    Node root = getTrie(); //   
    int N = BinaryStdIn.readInt(); // number of chars in string  
    for(int i=0; i<N; i++){  
        Node x = root;  
        while (!x.isLeaf()){  
            if(BinaryStdIn.readBoolean()) x = x.right;  
            else x = x.left;  
        }  
        BinaryStdOut.write(x.ch);  
    }  
    BinaryStdOut.close();  
}

output trie
ie. to serialize a trie.
⇒ use preorder traversal:
0 for internal nodes, and 1 for leaf (followed by the corresponding char)

recursive method:

private static void writeTrie(Node x){  
    if(x==null) return;  
    if(x.isLeaf()){  
        BinaryStdOut.write(true); // leaf node  
        BinaryStdOut.write(x.ch); // followed by the char  
    } else {  
        BinaryStdOut.write(false); // internal node  
        writeTrie(x.left);  
        writeTrie(x.right);  
    }  
}

read trie
Reconstruct the trie from the serialized bitstring.
preorder(x) = x+preorder(x.left)+preorder(x.right) ⇒ recursive method...

这个递归函数还是蛮有代表性的, 值得看.

private static Node readTrie(){  
    if(BinaryStdOut.readBoolean()) {  
        char ch = BinaryStdOut.readChar();  
        return new Node(ch, 0, null, null);  
    } else {  
        Node left = readTrie();  
        Node right = readTrie();  
        return new Node('\0', 0, left, right); // subtle recursion  
    }  
}

Huffman algorithm

pb: how to find the best prefixless code?

Shannon-Fano algo:

partition symbols S into 2 subsets: S1, S0, with roughly equal freq

code in S1 start with 1 and in S0 start with 0

recur on S1 and S0

[Huffman algo] (1950)

count letter freq in input

build a node for each char

select the 2 tries with minimum weight(freq), merge them, and put it back

repeat until we get only 1 trie

application: jpeg, pdf, mp3, ...

implementation

private static Node buildTrie(int[] freq){  
    MinPQ<Node> pq = new MinPQ<Node>();  
    for(char i=0; i<R; i++)  
        pq.insert( new Node(i, freq[i], null, null) );  
    while(pq.size()>1){  
        Node x = pq.pop(), y = pq.pop();  
        pq.insert( new Node('\0',x.freq+y.freq, x, y);  
    }  
    return pq.top();  
}

prop. Huffman algorithm produces the optimal prefixless code.

running time: N + RlgR

4. LZW Compression

idea:
adaptive model: learn and update the model as you read the text.
decoder will do the same thing.

build a table of not only mapping chars to codes, but also mapping >=2-char combinations to codes(of fixed width).
when encoding strings, look for the longest prefix that is in our table.

LZW compression

LZW compression algo

create a symbol table mapping string keys to W-bit code, initialize as mapping only chars to codes
find longest prefix s of the unscanned input
write out the corresponding code
add s+c into the symbol table, where c is the next char in input

Use a trie for representing the code table → because support longest prefix match.

implementation

public void compress(String input){  
    TST<Integer> st = new TST<Integer>(); // a trie symbol table  
    for(char i=0; i<R; i++)  
        st.put(""+(char)i, i);  
    int code = R+1; // R is used as "stop"  
    while(input.length()>0){  
        String s = st.longestPrefixOf(input);  
        BinaryStdOut.write(st.get(s), W);  
        int l = s.length();  
        if(l<input.length())  
            st.put(s+input.charAt(l), code++);  
        input = input.substring(l);  
    }  
    BinaryStdOut.write(R, W); // write "stop"  
    BinaryStdOut.close();

}

LZW expansion

no need to get the codeword table, the input is just the compressed bitstring.

maintain a (reverse)table mapping int(code) to strings.
as we decode the string, we add new entries to our table.

to represent the table: just an array.

note when add new entries: add key = s+c, where s is the last decoded string, c is the first char of the currently decoded string.

example: when decoded ABR, s = BR, currently decoded string = ABR, so we add BRA.

tricky case: ABABABA
compression

problem in expansion when reading the "83":

→ need to be able to update the table when encountering a code not yet defined...

Summary

Huffman: fixed-length symbols, with variable-length codes.
LZW: variable-length symbols, with fixed-length codes.

theoretical limit:

还剩下一周的内容, 可能要等到二月底考完以后有时间再看了, 现在先复习备考...

[Algorithms II] Week 5-1 Regular Expressions

2015-12-27T00:00:00+01:00

1. Regular Expressions

pb: pattern matching.

regular expression

Is a notation to specify a set of strings.
basic operations:

concatenation
or
closure: "0 or more appearances of chars"
parentheses

additional operations (added for convinence):

ex. [A-C]+ is equivalent to (A|B|C)(A|B|C)*.

吐槽名句:

2. REs and NFAs

duality between RE and DFA:

RE: to decribe a set of strings.
DFA: machine to ecognize whether a string is in a given set.

[Kleene's therom]

For any DFA, there exists a RE that describes the same set of strings;
For any RE, there exists a DFA that recognizes the same set of strings.

first attempt of pattern matching

(Ken Tompson) same as KMP — no backup.
basic plan:

construct the DFA
simulate the DFA with text

bad news: DFA may have exponential nb of states.
⇒ change to NFA (nondeterministic finite automaton).

NFA

put RE into parentheses
every char as a state (start=0, success=M) — 这里和之前的DFA很不一样: 之前是每个transition(edge)关联一个char, 这里是每个状态(node)关联一个char.
epsilon-transition (red links below): change of machine state without scanning text
match-transition (black links below): change state, but also have to scan next char in text, match transition is added after each alphabetic char
success (accept) if any sequence of transitions (after scanning all text) end at state-M.

亦可理解为, DFA是每一条边对应一个可能的(字母表内的)char, 而NFA只有match-transition对应于pattern里的(alphabetic) char, 其他epsilon transition的边对应空字符串(也就是epsilon string).

example:
is "AAAABD" a match ?
→ yes. (和上一节substring的插图进行一下比对, 还是有很大不同)

pb: non-determinism
How to determine whether a string is a match of a NFA (ie. how to select the right sequence of transition) ?
⇒ sysematically consider all possible transition sequences.

3. NFA Simulation

state names: 0 to M. (M+1 states in total, M=length of RE string).
match-transitions: store in array re[] (the match transitions are naturally in order of the array).
epsilon -transitions: store in a digraph G

idea:

maintain a set of all state that NFA could be in after reading first i chars in text.

at each iteration: check all reachable state wrt the transitions, then update reachable states.

algorithm

(for the NFA above, 注意为了方便已经加了必要的括号)

[Algo]

initial: rs(reachable state)=reachable state from state 0 (left parenthese) using epsilon trantisions

consume a char in text:

nrs (new-reachable-states) = empty set

from all reachable state of this character: add next state using the match-transition to nrs

add all reachable states (using epsilon transition) form the nrs set to nrs

set rs = nrs, and consume the next char in text

accept if at the end the state M is in rs

concrete example

init:

when matching A from text: state 2 or 6

using match transition of A, we can get to state 3 or 7

if we add epsilon transitions:

so reachable states after reading 1st A are: 2, 3, 4, 7

matching 2nd A from text: state 2

using match transition we can only get to state 3.

using epsilon transitions from state 3:

(the only state after matching A is state 2 3 4)

etc...

或者直接看这张图:

reachability

All reachable vertices from a set of source vertices → just DFS.
⇒ directly use the API from the digraph section:

running time linear to E+V

Java implementation

API:

public class NFA{  
    private int M;  
    private char[] re;  
    private Digraph G;// digraph of the epsilon-transitions  
    public NFA(String regexp){  
        M = regexp.length();  
        re = regexp.toCharArray();  
        G = buildEpsilonTransitionGraph();// helper function to build the graph G  
    }  
    public boolean matches(String text);// does text match the regexp?  
    private Digraph buildEpsilonTransitionGraph();// private helper function  
}

The function buildEpsilonTransitionGraph()will be attacked in next session, for now we focus on the NFA simulation code — that is, the mathes() method.

For simplicity let's assume we have a function reachableVertices(Digraph G, Bag<Integer> sourceSet) and reachableVertices(Digraph G, int source) that gives the reachable states from (a set of) source vertices, including the sources. Or we can directly use the DirectedDFS api as listed above.

    public boolean matches(String text){ //does text match the regexp?  
        Bag<Integer> rechableStates = reachableVertices(G,0);// init reachable   
        for(char c: text){  
            Bag<Integer> newRechableStatesBymatch = new Bag<Integer>();  
            for(int i:rechableStates)  
                if(re[i]==c || re[i]=='.')   
                    newRechableStatesBymatch.add(i+1);// match transition  
            rechableStates = reachableVertices(G,newRechableStates);//epsilon transition  
        }  
        return reachableStates.contains(M);  
    }

(代码虽然短但是这个过程我理解了好久.. 另外上面的代码有点伪).

Analysis

prop. the matches() method takes O(MN) time in worst case.

pf. N chars in text, each char can go through <= M states (DFS), and in the digraph, no node has >3 degree ⇒ number of edges <= 3M, so the time for each dfs is O(M), in total we have O(MN).

4. NFA Construction

→ construct the epsilon transition digraph.

buiding a NFA from a re (parsing)

states in a NFA: one state per char, plus an accept state (state M)
alphabet state: chars in alphabet (A, B, C, D) → (implicitly) put a match transition to next state
metacharacters: ( ) . * | , 5 metacharacters in total

⇒ to deal with the metacharacters:

paretheses ( )

simply put a epsilon-transition to the next state

closure *

星号前面只可能是字母(包括.)或者右括号), 所以分两种情况讨论一下, 需要向前看一位, 这里就比较subtle
for each * state, add 3 transitions as below:

or |

or符号肯定在一个括号里面
add 2 epsilon transitions wrt parethese:

以上就是NFA建立G的时候要处理的三种情形, 这三种情形都要知道一个左括号(lp)的位置 ⇒ use a stack !

implementation

for alphabetic chars: do one-char lookahead → if next is *, add transitions.
for left parenthese (: add transition to next state, and push to stack
for or |: add transition to next state, and push to stack
for right parenthese ): pop the stack to deal with or and lp; and also do lookahead.

code is not trival... look carefully:

private Digraph buildEpsilonTransitionGraph(){// private helper function  
    Digraph G = new Digraph(M+1);  
    Stack<Integer> stk = new Stack<Integer>();  
    int lp;  
    for(int i=0; i<M; i++){  
        if(re[i]=='|' || re[i]=='(')   
            stk.push(i);  
        if(re[i]=='(' || re[i]==')' || re[i]=='*')   
            G.addEdge(i,i+1);  
        else if(re[i]==')'){// need to pop until get a lp  
            int j = stk.pop();  
            if(re[j]=='|'){  
                lp = stk.pop();  
                int or = j;  
                G.addEdge(lp, or+1);// add edge for the `or` case   
                G.addEdge(or, i);

            }  
            else lp = j;  
        }  
        // do the lookahead:   
        if(re[i+1]=='*'){  
            if(re[i]==')'){ // case 1 of closure: a rp before `*`  
                G.addEdge(lp, i+1);  
                G.addEdge(i+1, lp);  
            }else{ // case 2 of closure: an alphabetic char before `*`  
                G.addEdge(i, i+1);  
                G.addEdge(i+1, i);  
            }  
        }  
    }// go through each char in re  
    return G;  
}

Analysis

prop. building an NFA takes linear time and space in M.
pf. for each char, the nb of operations is const.

真不愧是most ingenius algorithm we met in this course......

5. Regular Expression Applications

grep

"Generalized Regular Expression Print"
print out all lines (from stdin) having a substring of an RE.
⇒ equal to adding a .* to the beginning and end of the RE to make a match.

public class GREP{  
    public void main(String[] args){  
        String re = ".*"+args[0]+".*";  
        NFA nfa = new NFA(re);  
        while(StdIn.hasNextLine){  
            String line = StdIn.readLine();  
            if(nfa.matches(line)) StdOut.println(line);  
        }  
    }  
}

the grep has NM worst case running time — same as brute force substring search — amazing...

grep application: crossword puzzles

regexp in other languages

unix: grep, awk
script: python, perl

java: String.matches(regexp)

Harvesting information

goal: print all substrings of input that match an RE.
use Pattern and Matcher class in java.util.regexp.
first compile the regexp, then build the matcher
→ so that we can iterate through all matches of the input using find() and group() of the matcher

Caveat: performance NOT guaranteed !

→ exponential time growth!

Not-so-regular expressions

"not rugular" means Kleene's Th doesn't hold
→ efficient performance not tractable......
back-reference
\1 matches subexpressions that was matched earliser
limitations of regular languages:

Summary

the substring and regexp are examples of compilers ! (from string to a NFA/DFA/bytecode)

Priority Queue/Heap (优先队列/堆)小结

2015-12-09T00:00:00+01:00

今天简单介绍一下优先队列(priority queue, 以下简称PQ)这个数据结构的实现.

PQ又叫"堆"(heap), 但是可能优先队列这个名字更容易记忆它的用途: pq是一种队列, 不过不是先进先出(FIFO), 而是每次出队的元素永远是优先级最高的.

logical structure of PQ

个人感觉"堆"这个名字大概源于PQ的(逻辑上的)形状吧: PQ是一种树(tree), 准确的说, 是一种二叉树(binary tree), 说得再准确一点, 它是一种完全二叉树(complete binary tree): 没错, PQ是一种满足某些条件的完全二叉树.

所谓的"完全二叉树", 要满足:

除了最后一层, 所有层都排满(没有非空节点)

最后一层的所有非空节点都排在左边

一张图可以直观说明, 完全二叉树其实就是长得像这样:

一个完全二叉树能被成为PQ的话, 要满足的条件就是:

对于任何一个节点, 它的优先级都大于左右子节点的优先级.

比如下图(圆圈里数字代表优先级):

这样看上去的话, 确实像是"一堆"东西的形状(码的还挺整齐的), 而且我们知道在这一堆节点里, 优先级最高的就是最顶上的那个节点了.

我很喜欢这个比喻: 一个heap就像是一个公司的hirachy结构: 子节点就是下属, 父节点就是上司, 每一个上司的能力都比他的下属强(优先级比子节点搞). 能力最强(优先级最高)的人在最上面.

array representation of PQ

对于一个完全二叉树, 没有必要用常规的树结构(使用指针)来表示, 因为如果从上到下走过每层(每层内从左到右)给所有节点编号(根节点的编号为1)的话, 完全二叉树有以下性质:

father(i) = i/2 其中father(i)表示编号为i的节点的父节点的下标

leftchild(i) = i*2, rightchild(i) = i*2+1

既然用数组表示的时候, 任何节点的父亲节点和左右子节点都可以轻松得到, 就没有必要使用指针了. 所以只需要一个数组即可表示PQ ! 比如一个int的PQ在内部只要表示为: int pq[].

另外注意, 上面的公式成立的前提是数组下标从1开始, 实现的时候我们把数组的第0个元素空出来即可.

不难看出一个PQ有以下性质:

高度为lgN

第k层有 2^k 个节点 (root是第0层)

最后一层的节点对应的下标>=N/2

PQ implementation

先写一下代码框架:

public class PQ{// maxPQ of integers  
    int pq[];  
    int MAX_CAPACITY=1000;// if we use ArrayList we do not need MAX_CAPACITY  
    int size;// pq[size] is the index of last element (rightmost node in last level)   
    public PQ(){  
        pq = new int[MAX_CAPACITY+1];  
        size = 0;  
    }  
    public boolean isEmpty(){  
        return size==0;  
    }  
    public int top(){ // get top element  
        assert !isEmpty();  
        return pq[1];  
    }  
    public void add(int n); // insert element to PQ --> stay tuned  
    public int poll(); // get and remove top element --> stay tuned  
}

最关键的两个函数是add和poll, 实现这两个操作的关键在于两个基本操作: siftup()和siftdown(). 在PQ的性质被扰乱的时候, 使用这两个操作可以使PQ恢复性质(所谓的"调整").

siftup

如果一个新元素到来, 首先将其放在最后(最底层最右边的节点), 如果新来节点的优先级比较高, 可以用siftup将其提升到一个合适的层次. 用公司结构来类比就是新入职的员工由于能力强被提高到了高层, 而提升的办法就是: 和上司交换岗位(也就是和父节点交换位置).

所以这个操作只要不断把该元素与父节点交换, 直到遇到一个优先级大于该节点的父节点或者已经到了最高处停止. 代码很短:

private void siftup(){  
    int i = size;// i is the index of the newly added element     
    for(;i>1 && pq[i/2]<pq[i]; i/=2)  
        swap(pq, i/2, i);  
}

siftdown

siftdown的功能和siftup相反: 如果在最高处是一个优先级很低的元素, 需要将其"下放". 方法就是把它和子节点里面优先级较高的进行交换.

代码也不长, 不过第一次写可能不太简洁:

private void siftdown(){  
    if(isEmpty()) return;// nothing to sift when empty  
    int i=1;// siftdown root node  
    while(i*2<=size){ // while the node is not in last level  
        int max=pq[i], j=i;// j is the element to swap  
        if (pq[i*2]>max) // left child  
            {j=i*2; max=pq[i*2];}  
        if (i*2+1<=size && pq[i*2+1]>max) // right node  
            {j=i*2+1; max=pq[i*2+1];}  
        if (j==i) return;// stop when node is bigger than both child  
        swap(pq, i, j);  
        i = j;  
    }  
}

implementation of add and poll

有了这两个sift的基本操作, add和poll的操作就简单了.
add
加入新元素的时候, 只要先把它放在最后面, 然后调用siftup()函数调整一下pq即可:

public void add(int n){ // insert element to PQ  
    assert size+1<MAX_CAPACITY;  
    pq[++size] = n;  
    siftup();  
}

poll
堆顶元素被拿走以后, 我们可以先把最后一个元素放在顶部, 然后调用siftdown进行调整:

public int poll(){ // get and remove top element  
    assert !isEmpty();  
    int top = pq[1];  
    pq[1] = pq[size--];// put last element on the top  
    siftdown();  
    return top;  
}

PQ application

接下来讲一下PQ的应用.

heapsort

由于每次出队的都是在剩下元素里面最大(小)的, 所以只要把数组的元素放到一个pq里, 然后依次poll出来, 得到的序列就是排序好了的.

不管是插入还是删除操作, 每次调整的复杂度为log(n) (堆的高度), 所以算法复杂度为 O(NlgN). 实际使用的时候效率比快速排序/合并排序略差, 以后专门写一篇关于排序算法的文章时再聊.

heapify

首先前面的siftdown函数很明显可以修改一下加上一个参数变为siftdown(int i)(i代表要向下调整的元素的下标).

在heapsort里第一步是要建立一个PQ. 最naive的建堆操作就是新建一个空的heap然后不断向里面加入数组a里的元素(空间复杂度N, 时间复杂度NlgN). 其实这个操作可以做的更好: 我们先直接把数组a当作pq[]数组, 现在显然它不满足PQ性质, 只需要多次使用siftdown进行调整即可.

假设一共有h(=lgN)层, 由于最后一层的节点不必调用siftdown, 我们只要从倒数第二层开始调用siftdown即可, 结合前面提到的pq的性质(N/2以后的节点都在最后一层), 写法很简单(简单起见认为a也是把第0个元素空出来好了):

public static void heapify(int[] a){  
    int N = a.length;  
    for(int i=N/2;i<=1;i--){  
        siftdown(a, i);  
    }  
}

这个操作的时间复杂度是O(N)的 ! 为什么呢?
→ 第k层节点有2^k个节点, 这一层的节点向下调整最多会进行h-k步, 所以计算量是一个求和表达式:
Sigma( 2^k * (h-k) ) for k=0,...,h-1

(具体见 http://algs4.cs.princeton.edu/24pq/ 里面Q20的答案)

top K elements of a stream

问题描述: 一系列数字一个一个到达, 当所有数(假设有N个)都到达以后, 求最大的K个数字.

这是一个经典的PQ应用 ⇒ 使用一个最小堆(minPQ)即可达到这个效果: 新建一个minPQ(最小的数字在最上面, 每次当一个数字n到达时:

if minPQ.size<K → minPQ.add(n)
if minPQ.size==K → 比较n和minPQ.top():
- if n>minPQ.top(): minPQ.poll(); minPQ.add(n)
- else pass...

median of a stream

问题描述: 一系列数字一个一个到达, 求他们的中位数(median)?
这个的解法是使用两个PQ, 一个最大堆maxpq一个最小堆minpq, 然后maxpq里存放较小的一半数字, minpq里存放较大的一半数字: maxpq.top()<=minpq.top().
并且我们要求: minpq.size <= maxpq.size <= minpq.size+1
⇒ 这样中位数或者是maxpq.top(), 或者是两个top的平均值了~

初始时两个pq都是空的, 每次数字n到来时:
先把n放进maxpq里
如果maxpq.size>minpq.size+1的话: minpq.add(maxpq.poll())

more about heap

k-way heap

这里介绍的PQ其实是binary heap, 即这棵树是一个完全二叉树, 但是我们完全可以做成完全K叉树, 就是所谓的K-way heap了.

这样的好处是heap的高度会变得更小(从log2(N)变为logK(N)), 不过由于每个分叉变成了K个, 所以siftdown每次循环的操作复杂度也会增加(从2变为K).

more "flexible" heap

在某些应用中(比如Dijkstra算法), 我们希望在插入后修改某个元素的优先级, 这时候需要对这个标准的heap做一些修改, 需要用一个数组把元素的"逻辑编号"和在pq数组里的"实际编号"相互转换.

不过这个会比较subtle(其实我已经记不起来了...), 见: http://x-wei.github.io/algoII_week2_2.html (IndexedPQ部分)

[Algorithms II] Week 4-2 Substring Search

2015-12-07T00:00:00+01:00

1. Introduction to substring search

"most ingenious algorithm we've seen so far"
pb. having two strings, pattern and text, len(pattern)=M << len(text)=N, try to find pattern in text.

ex. indexOf method of String in java.

2. Brute-Force Substring Search

function signature:
public static int search(String pat, String txt);

brute-force algo: look for pattern at every position of text.

public static int search(String pat, String txt){  
    int N=txt.length(), M=pat.length();  
    for(int i=0; i<=N-M; i++){  
        int j;  
        for(j=0; j<M && pat.charAt(j)==txt.charAt(i+j); j++);  
        if(j==M) return i;  
    }  
    return N;// not found  
}

worst case: when txt/pat are repetitive → MN compares.

problem with brute-force: always backup when mismatch.

brute-force alternative

j := number of matched chars in pattern
i := index of the end of matched char in text

⇒ do explicite backup when mismatch by i -= j

public static int search(String pat, String txt){  
    int N=txt.length(), M=pat.length(), i=0, j=0;  
    while(i<N && j<M){  
        if(pat.charAt(j)==txt.charAt(i))   
            {j=i++; j++;}  
        else   
            {i=i-j+1; j=0}// <==backup  
    }  
    return j==M ? i-M : N;  
}

challenge: want linear-time guarantee, and want to avoid backup.

3. Knuth-Morris-Pratt

"one of the coolest/trickiest algorithm covered in this course"

intuition

suppose pattern = "BAAAAA",
if we matched 5 chars in pattern and get mismatch on 6th char ⇒ we know the previous 5 chars are "BAAAA" → no need to backup the i pointer.

KMP algorithm: clever method that always avoid backup !

Deterministic finite state automaton (DFA)

finite states ,including start and halt state, indexed by j in the subtring pb
for each state: exactly one transition for each char in alphabet

ex.
states are 0~6, pat="ABABAC", transitions are indexed by chars in alphabet = {A,B,C}, finish if we reach state-6.

dfa[c][i] = the next state if we are currently in state-i and encoutered char-c.

interpretation of DFA for KMP algo

in the DFA after reading txt[i], the index of state is the number of matched chars in pattern, or length of *longest prefix of pat that is a suffix of txt[0:i]. *

need to precompute the dfa[][] array from pattern
the pointer i never decrements (thus we can do it in a streaming manner)

→ if dfa[][] is precomputed, java code is very very simple:

public static int search(String pat, String txt, int[][] dfa){  
    int N=txt.length(), M=pat.length(), i, j=0;  
    for(i=0; i<N && j<M; i++)  
        j = dfa[txt.charAt(i)][j];  
    return j==M ? i-M : N;  
}

running time: linear.
→ key pb: how to build dfa efficiently ?

DFA construction

match transition (easy part)

when at state j, for the char c0==pat.charAt(j+1), just go on matching: dfa[c0][j] = j+1
ex. (pat="ABABAC")

mismatch transition (hard part)

(for j==0, things are simple: dfa[c][0]=0 for all c!=pat[0])

at state j (ie. j chars in pattern are matched), and for c!=pat.charAt(j+1)

⇒ we are in state j: we know the last j chars in input are pat[0...j-1], and followed by char = c, so the last j+1 chars of input string is: pat[0...j-1]+c

⇒ to compute dfa[c][j]: we can simulate as if we backup, ie. i=i-j+1, j=0.

if we go back to set j=0, and set i = i-j+1, then i is pointing at pat[1], the text become pat[1...j-1]+c. We then let this string go through our dfa, the state that it achieves is the value of dfa[c][j].

here is a concrete example:
pattern = "ABABAC", state j=5, char c='B'

we know the last 6 chars of the input = pat[0...j-1]+c="ABABA"+"B"="ABABAB"

if we backup, i will point to pat[1], the string is just pat[1...j-1]+c="BABAB"

we use the string "BABAB" as input and go through the partially constructed dfa, and see that we will reach state 4

so we know dfa['B'][5]=4

similarly we can get dfa['A'][5]=1, as indicated below:
(pat="ABABAC")

one concern: seems this simulation needs j steps ?
⇒ can be changed to be constant time if we maintain a state X := the state of simulating of input=pat[1...j-1]
we maintain this state X, then for each mismatched char c, we just need to look at dfa[c][X].
(pat="ABABAC")

[Algo]

set all matched transitions dfa[c0][j] = j+1 for all c0==pat[j]

fill first column (j==0): dfa[c][0]=0 for all c!=pat[0]

initialize X=0 (state for empty input string)

for j=1 to M:

for all c!=pat[0]: set dfa[c][j] = dfa[c][X] (DP here...)

update X=dfa[c0][X] ⇒ 注意, 此时X并不等于X+1(最开始dfa[c0][j]=j+1不适用于此), 为什么? 因为c0==pat[j] 而不是pat[X]!! 比如说最开始, j=1的时候X是等于0的!!! (这个弯我饶了好几分钟...)

java code (can be written to be more compate):

public int[][] constructDFA(String pat){  
    int R=256;//ASCII code    
    int M=pat.length();  
    int[][] dfa = new int[R][M];  
    // 1. fill matched transitions: dfa[pat.charAt(j)][j] = j+1   
    for(int j=0;j<M;j++)  
        dfa[pat.charAt(j)][j] = j+1;  
    // 2. fill 1st column --> can be ignored as java int default val=0  
    // 3. fill mismatched transitions     
    int X = 0;  
    for(int j=1;j<M;j++){  
        for(int c=0;c<R;c++)  
            if(c!=pat.charAt(j))  
                dfa[c][j] = dfa[c][X];  
        X = dfa[pat.charAt(j)][X];  
    }  
    return dfa;  
}

running time and space: O(M*R).

prop.
KMP algorithm runs in O(M+N) time, and constructs the dfa in O(M*R) time/space.

这个KMP算法, 我曾经想过好几个小时, 然后最后写出了特别复杂的代码, 虽然可以用但是基本写了就忘掉了. 但是经过老爷子这么一讲, 感觉这次印象深刻了好多. 老爷子NB...

八卦时间:

4. Boyer-Moore

Heuristic in practice.
i does not necessarily go through all txt chars ⇒ i may skip some chars.

intuition

for matching: scan chars from right to left (j will decrease when checking)
when encoutered a mismatch: we can skip <= M chars (if the char is not in pattern)

ex. (pat="NEEDLE")

→ pb: how to skip?

mismatch character heuristic

note: the i always points to the beginning of the substring (txt[i,...,i+M-1]) to be checked for match.

case 1. mismatched char not in pattern
easy case → just move i to the right of this char.

case 2. mismatched char in pattern

heuristic: line up i with the rightmost char in pattern.
i += skip
where skip length = j - index of rightmost char in pattern

note: this does not always help, in the example below, i even backups:

to avoid backup, in this case we just increment i by 1 (heuristic doesn't help in this case).

implementation

use an array right[] as skip table, right[c] is the index of rightmost occurrence of char c (-1 if c not in pat).

int[] right = new int[M];  
for(int i=0;i<R;i++) right[i] = -1;//value for chars not in pattern  
for(int j=0;j<M;j++){  
    right[pat.charAt(j)]=j;  
}

using this table we can implemente the heuristic algorithm:

public static int search(String pat, String txt, int[] right){  
    int N=txt.length(), M=pat.length();  
    int skip;  
    for(int i=0;i<N-M;i+=skip){  
        skip = 0;  
        for(int j=M-1;j>=0;j++)  
            if(pat.charAt(j)!=txt.charAt(i+j)){// when mismatch happens  
                skip = Math.max(1,j - right[txt.charAt(i+j)]);// skip if we can, else just increment i by 1  
                break;  
            }  
        if(skip==0)// if the above for loop finishes without changing skip --> we are done.    
            return i;  
    }  
    return N;// pattern not found  
}

analysis

property. the Boyer-Moore heuristic (in practice) takes about N/M (sublinear!) compares to search.

好神奇, 比KMP还要简单的算法, 实际效率这么高...

worst-case performance: N*M... 这一点不如KMP.

→ can be improved...

5. Rabin-Karp

两个图灵奖的大神发明的算法..

intuition

basic idea: modular hashing
ex. for strings of numbers

compute hash fcn (for number strings is easy: take the string and treat it as a number, then %Q where Q is a big prime number).
for a pointer i →corresponds to the substring txt[i, ..., i+M-1] → check hash for match

(below: text=3141592653589793, pattern=26535)

computing the hash function efficiently

let ti be the ith char in txt, the hashcode for substring txt[i,...,i+M-1] is:

⇒ just an M-digit base-R integer modulo Q ! poly(M, R) % Q*. *

Honor's method

linear time algorithm for evaluating polynomial.
recursive equation: poly(i, R) = poly(i-1, R)*R+ti

ex. (R=10, M=5)

private long hash(String key, int M){  
    long h=0;  
    for(int i=0;i<M;i++)  
        h = ( h*R + key.charAt(i) ) % Q  
    return h;  
}

if we know x_i, the x_i+1 can be infered:

⇒ x_i+1 can be computed in constant time:

⇒ we precompute R^(M-1) and maintain the hash number, and check for match !

public static int search(String txt, String pat){  
    int N=txt.length(), M=pat.length();  
    long pathash = hash(pat, M);      
    int RM = R^(M-1);// <-- pseudo code, store value of R^(M-1)  
    long txthash = hash(txt, M);// txthash will be maintained  
    for(int i=0;i<N-M;i++){  
        if(txthash==pathash && checkMatch(i,txt,pat))   
            return i;  
        txthash = ( (txthash - txt.charAt(i)*RM)*R + txt.charAt(i+M) ) % Q;  
    }  
}

更新txthash的地方可能会有modulo造成的问题... 不过先这样写吧..

for collisions: Monte Carlo vs. Las Vegas

analysis

Theory: if Q is sufficiently large (~MN^2), the probability of collision is ~1/N.
Practice*: choose Q to be sufficiently large, and collision probability is ~1/Q.

Summery

quick sort and more: 快速排序算法总结

2015-12-07T00:00:00+01:00

今天总结一下非常有用的快速排序(qsort)算法, 以及由此衍生的一些其他相关算法(Knuth shuffle, quick select, 3-way partition).

快速排序的算法可以用三句话描述:
[Algo]

选择基准项(pivot element, 一般取第一个元素为pivot)

把数组里所有小于pivot的移动到pivot左边, 大于pivot的移动到右边 ⇒ 此时pivot已经位于最终排序时的正确位置

对pivot左右两个数组分别递归进行快速排序

由以上描述可见, qsort是一个递归算法, 我们可以把它的函数声明写成: void qsort(int[] a, int lo, int hi), 表示排序a[lo, hi]之间(闭区间)的所有元素.

quick partition

由上面描述可以见, qsort最关键的是第二步: 把数组元素以pivot分为两部分. 这个操作就是quick partition.

函数声明为: int partition(int[] a, int lo, int hi), 该函数返回pivot(即subarray的第一个元素a[lo])所在的位置.

如果允许新建一个临时数组的话, 那么这个就不是什么问题, 但是为了节约空间占用, 现在需要直接修改(in-place)使得a[lo] 到, 而且希望可以用尽量少的交换(swap(int[]a, int i, int j))操作, 就不是很evident了.

这个函数的写法是用两个指针i和j分别从两端向中间走, 如果两个指针指向的元素一个小于pivot一个大于pivot那么就进行交换, 当两个指针碰面的时候结束(最后把pivot和指针元素交换). 请看下面这个萌萌的图(图片来自<<啊哈!算法>>):

选取第一个元素(6)为pivot, 然后j向左走直到遇到一个小于pivot(6)的数停止, i向右走直到遇到一个大于pivot的数停止(注意要让j先移动), 此时二者交换:

只要重复这个过程, 直到i>=j为止, 此时只要最后把pivot和j(注意是j而不是i)指向的元素交换即可:

所以pivot的位置就是j, 函数返回j即可.

java实现:

int partition(int[] a, int lo, int hi){  
    int pivot = a[lo], i=lo, j=hi;  
    while(true){  
        for(;j>=lo && a[j]>=pivot;j--);// move j to a point where a[j]<pivot  
        for(;i<=hi && a[i]<=pivot;i++);// move i to a point where a[i]>pivot  
        if(i>=j) break;// break if i and j meets  
        swap(a, i++, j--);  
    }  
    swap(a, lo, j);// swap pivot with a[j]  
    return j;  
}

这里有几点要注意的:

让j先移动
最后pivot要和j交换而不是和i交换: 因为最后放在最左边的应该是一个小于pivot的数嘛
移动的时候别忘了需要加数组下标的边界检查(i<=hi, j>=lo)
partition()的复杂度是线性的O(n)

Knuth shuffle

qsort之所以快, 是因为每次都能够按照pivot分为大致同样长度的两个子数组(所以每次子问题的规模除以二), 所以复杂度为O(NlogN). 最坏情况下, 如果每次两个子数组中可能有一个长度为0, 那么每次子问题的规模只减少了1, 所以复杂度变成了quadratic O(N2).

为了防止这种最坏情况的出现, 可以在一切开始之前把数组打乱顺序, 所以这一节讨论快速shuffle的算法. 最经典的就是Knuth的shuffle算法了, 算法很简答, 描述为: for(k=1 to n): 每次把第k个元素和前k个元素中的随机一个元素交换.

代码只有两行:

void shuffle(int[] a){  
    for(int K=0; K<a.length; K++)  
        swap(K, Random.nextInt(K+1));  
}

关于算法的正确性, 其实只要证明"元素i在shuffle后最终位于位置i"的概率为1/N即可, 不难证明.

quick sort

好了有了以上两个辅助函数就可以写qsort函数了:

void qsort(int[] a, int lo, int hi){//recursive helper function  
    if(lo>=hi) return;  
    int p = partition(a, lo, hi);  
    qsort(a, lo, p-1);  
    qsort(a, hi, p+1);  
}  
void qsort(int[] a){  
    shuffle(a);  
    qsort(a, 0, a.length-1);  
}

其实qsort的主体就是那个partition函数, 单独把partition列出来是因为它不止可以用在排序, 还可以用来做quick select, 见quick select节.

quick select

对于一个没有排序的数组, 如何快速找到它的中值(median)?
以上这个问题的答案就在partition()函数.

之前说过, partition()函数的返回值表示pivot在排序好的数组中的位置(rank), 这个消息非常有用: 中值只不过是rank等于长度除以2的元素而已.

为了寻找rank等于k的元素, 我们用partition函数可以每次把问题规模缩小: 如果partition()=pk那么右边subarray不用考虑, 如果数组事先shuffle过了的话, 问题规模每次缩小一半.

定义一个函数, 寻找rank等于k的元素, 代码类似于二分查找:

int findKth(int[] a, int k){  
    shuffle(a);  
    int lo=0, hi=a.length-1;  
    while(lo<hi){  
    int p = partition(a, lo, hi);  
    if(p==k) return a[k];  
    else if(p<k) lo=p+1;  
    else hi=p-1;  
    }  
    return a[k];  
}

该算法内层循环为O(hi-lo), 每次问题规模减少一半, 所以复杂度为N+N/2+N/4+...+1 = 2N, 复杂度为线性时间!

3-way qsort

qsort之前有个bug: 在数组里很多重复元素的时候, 效率会下降为O(N2). 原因是qsort没有好好处理重复元素的问题.

于是Dijkstra提出了一个3-way partition的算法: 把数组分为三部分: 左边[lo, lt)严格小于pivot, 中间[lt, gt]等于pivot, 右边(gt, hi]严格大于pivot.

算法初始化lt=lo, gt=hi, i=lo, 用指针i向右扫描, [i,gt]为未处理到的部分.
算法很subtle, invariant是这样的:

a[lo,lt-1] < pivot

a[lt, i-1] = pivot

a[i,gt] = unseen

a[gt+1, hi] > pivot

这个图很有助于写代码:

void qsort3way(int[] a, int lo, int hi){  
    if(hi<=lo) return;  
    int lt=lo, i=lo, gt=hi, pivot=a[lo];  
    while (i<=gt){// [i,gt] is unseen elements  
        if(a[i]==pivot) //a[lt,i-1] are elements == pivot  
            i++;  
        else if(a[i]>pivot) // a[gt+1, hi] are elements > pivot  
            swap(a, gt--, i);  
        else // a[lo, lt-1] are elements < pivot  
            swap(a, lt++, i++);  
    }  
    qsort3way(a, lo, lt-1);  
    qsort3way(a, gt+1, hi);  
}

说它很subtle, 除了因为没有那个图我写不出来以外, 还有就是, 在把i和lt交换时, i可以increment (因为我们知道a[lt]==pivot), 但是i和gt交换时, i不能increment: 因为a[gt]不知道多大, 所以i位置要继续检查.

另外说一句, quicksort的思想在radix sort和trie里面也有闪现, 见我关于这两个主题的MOOC笔记...

[Algorithms II] Week 4-1 Tries

2015-12-05T00:00:00+01:00

More efficient version of symbol-table where the keys are strings.

1. R-way Tries

Two implementations of symbol tables that we've seen:

when keys are strings:
(L=string length, N=number of strings, R=radix)

for string keys ⇒ do better by avoiding examing the entire key.

goal: faster than hashtable, more flexible than BST !

String Symbol Table API

public class StringST<V>{// implements ST<String, V>  
    public void put(String key, V val);  
    public V get(String key);  
    public void delete(String key);  
}

R-way tries

A trie is a tree where:

except the root, each node store characters (instead of string keys) — in fact the chars are stored in links
each node has R children
store value in node if the node corresponds to the last char in key.

example: (a trie)

search in a trie get()

hit if when search ends the node has a non-null value
miss if reach a null link or have null value when search ends.

insertion in a trie put()

follow the links as the chars indicate in the key
→ if meet null links: create new node
→ when reach last char in key: set the value of the node

Java implementation

trie node class:
in each node use an array of size R to store links...

private static class Node{  
    private Object val;// because cannot create arries of generic type (array of Node)  
    private Node[] next = new Node[R];// chars are implicitly defined by link index  
}

另外关于为什么内部类Node声明为static, 参考这里:
http://www.geeksforgeeks.org/static-class-in-java/

implementation: use private recursive methods, 和BST的实现类似, 定义一个递归的private函数, 返回插入后的Node, 很有用.

public class TrieST<Value> {  
    pirvate final int R = 256; // ASCII chars  
    private Node root = new Node();  
    private static class Node{  
        private Object val;// because cannot create arries of generic type (array of Node)  
        private Node[] next = new Node[R];// chars are implicitly defined by link index  
    }  
    public void put(String key, Value val){  
        this.put(root, key, val, 0);// use private recursive helper method  
    }  
    private Node put(Node x, String key, Value val, int d){  
        // `d` is the index of char to put  
        // returns the inserted node

        if (x==null) x = new Node();  
        if (key.length()==d) x.val = val;  
        else{  
            char c = key.charAt(d);

            x.next[c] = put(x.next[c], key, val, d+1);  
        }  
        return x;  
    }  
    public boolean contains(String key){return get(key)!=null;}  
    public Value get(String key){  
        Node nd = this.get(root, key, 0);  
        if(nd==null) return null;  
        return (Value) x.val; // cast back  
    }  
    private Node get(Node x, String key, int d){  
        // returns the node that contains val for key  
        if(x==null) return null;  
        if(d==key.length()) return x;  
        char c = key.charAt(d);

        else return get(x.next[c], key, d+1);  
    }  
}

delete node in trie:

find the node and set the val to null
if a node has 0 links(leaf) and val==null: delete it and recursivly go up.

analysis

time:
search hit — L nodes examined.
serach miss — sublinear in L
space:
each node has R links (possibly null) — too much memory for large R !

Application
interview question: data structure for spell checking.

2. Ternary Search Tries

Solution to the memory issue of R-way tries — much fewer null links. ternery=tuple of 3...
TST:

store chars (and vals) explicitly in nodes
each node has 3 children:
- smaller, larger: TST that starts with char smaller/bigger than its char c.
- equal: the TST with keys that starts with this char c
- (所以除了用smaller/larger连接的节点在Rway trie里属于同一层

类似于3-way radix sort(同一篇paper里提出的), 有有点RBTree的意思...

searching in TST: quite similar to BST search, will go down ("equal" or "middle") when the current node matchs current char...

Implementation

Node class:

private class Node{  
    private Value val;  
    private char c;// store char explicitly  
    private Node left, mid, right;  
}

TST class (again use recursive helper functions)
有一种写BST代码的感觉, 另外由于这次char是显式存在node里, 代码反而更好理解了...

public class TST<Value>{  
    private class Node{...};  
    private Node root;  
    public void put(String key, Value val){  
        this.put(root, key, val, 0);  
    }  
    private Node put(Node x, String key, Value val, int d){  
        char c = key.charAt(d);// char to process  
        if(x==null) x = new Node(c);  
        if(c==x.c) {  
            if(d==key.length()-1) x.val = val;  
            else x.mid = put(x.mid, key, val, d+1);// d+1 means we go down one level  
        }  
        else if (c<x.c) x.left = put(x.left, key, val, d);// do not increment if the current char is not matchd  
        else x.right = put(x.right, key, val, d);  
        return x;  
    }  
    public Value get(String key){  
        Node nd = this.get(root, key, 0);  
        if(nd==null) return null;  
        return nd.val;  
    }  
    private Node get(Node x, String key, int d){  
        if(x==null) return null;  
        char c = key.charAt(d);  
        if(x.c==c) {  
            if(d==key.length()-1) return x;  
            else return get(x.mid, key, d+1);  
        }  
        else if(c<x.c) return get(x.left, key, d);  
        else return get(x.right, d);  
    }  
}

Hybrid of TST and Rway trie

at root: do R^2 branching
other nodes are TSTs

analysis

space cost: linear in N.
time: if keys arrive in rand order... (can use rotation to get worst-case guarantee..)

Hashing vs. TST

Hashing:

need to check entire key no matter hit or miss
performance relies on hash functions
donot support ordered operations

TST:

examines just enough chars
support ordered operations

3. Character-Based Operations

some very useful char-based operations:

prefix match
wildcard match: use a dot to represent any char
longest prefix: find the key that is the longest prefix of a string

API

public class TrieST<Value>{  
    // functions decalred before..  
    Iterable<String> keys;  
    Iterable<String> keysWithPrefix(String s);  
    Iterable<String> keysThatMatch(String s);  
    Iterable<String> longestPrefixOf(String s);
}

keys(): ordered iteration

keys(): just an inorder-traversal of the Rway trie →dfs, + maintain the chars in the path (root to current node).

public Iterable<String> keys(){  
    Queue<String> q = new Queue<String>();  
    collect(root, "", q);// helper fcn  
    return q;  
}  
private void collect(Node x, String pathstr, Queue q){  
    if(x==null) return;  
    if(x.val!=null)// if this is a key  
        q.enqueue(pathstr);// pathstr is the string of chars from root to x  
    for(char c=0;c<R;c++) // dfs  
        collect(x.next[c], pathstr+c, q);  
}

prefix match

very useful: ex. autocomplete, search bar, ...
implementation⇒ just find the end of that prefix, then call keys() on the subtrie.
之前写的俩helper function这里发挥作用了:

public Iterable<String> keysWithPrefix(String prefix){  
    Node subtrieRoot = get(root, prefix, 0);  
    Queue<String> q = new Queue<String>();  
    collect(subtrieRoot, prefix, q);  
    return q;  
}

longest prefix

⇒ just do a search and keep track of the longest key that we encounterd.

public String longestPrefixOf(String s){  
    int len = search(root, s, 0, 0);  
    return s.substring(0, len);  
}  
private int search(Node x, String s, int d, int len){  
    if(x==null) return len;  
    if(x.val!=null) len=d;// this is the current longest prefix  
    if(d==s.length()) return length;   
    char c = s.charAt(d);  
    return search(x.next[c], s, d+1, len);  
}

application: T9 texting

classique... G面试以及X的TP里都有涉及过...

first: generate all string combinations
second: call get() on the trie of dictionaries.

Other Variants

Patricia trie

improvement: remove the one-eay branching (put >1 chars in a node).

suffix tree

patricia tree of the suffix
linear time construction
ongest repeated substring, longest common substring, longest palindromic substring, substring search, tandem repeats... 好有用!!...

Summery

[Algorithms II] Week 3-2 Radix Sorts

2015-11-23T00:00:00+01:00

This week: string sort.

1. Strings in Java

char data type

char in C

8-bit integer, 256 characters, 7-bit ASCII code

char in Java

16-bit Unicode

String data type

String: immutable sequence of characters
operations: lengthe, ith char, substring, concatenate

implementation: using a char[], maintain a length and an offset. ⇒ substring methode is O(1) time.

StringBuilder data type

StringBuilder: mutable data type.
implementation: using a resizing char[] array (ArrayList).
⇒ contat in (amortized) constant time, substring in linear time!

ex.

reverse a string: linear using StringBuilder, quad using String.
form an array of suffixes: quad (time&space) using StringBuilder, linear (time&space) using String.

Longest Common Prefix:
runs in linear/subinear time ⇒ compareTo() for strings takes (sub)linear time!

Alphabet

alphabet different for different type of string (ex. binary numbers, DNA, ...)

def. Radix R is number of digits in alphabet.

2. Key-Indexed Counting

review of compare-based sorting algorithms:

lower-bound for compare-based algorithms: ~NlgN (=Lg(N!))
⇒ goal: do better by avoiding using compares.

Key-indexed counting

is an algorithm to sort by character(ex. sort array of string by their 1st character).
Assumption: keys are integers between 0 and R-1 (<<N, small integer).
⇒ use keys as array index, to sort an array of N integers between 0 and R-1.

[algo]

count freq of each key index (of size R+1)

using count array, compute a cumulated freq (cumsum of count[])

the cumsum[] array stores the index range of each key index:

index range of key-i in sorted array is [cumsum[i], cumsum[i+1]]

then get the sorted array by going through the array and using cumsum[] array

⇒

public void keyIndexCounting(int[] a, int R){// entries in a[] are in range [0,R-1]  
    int N = a.length;  
    int[] count = new int[R+1]; // count[i] = freq of key i-1, count[0] = 0  
    for(int ai:a)   
        count[ai+1]++;  
    int[] cumsum = count; //cumsum = cumulated freq  
    for(int i=0;i<R;i++)   
        cumsum[i+1] += cumsum[i];  
    // the ranges of key i in sorted array should be [cumsum[i], cumsum[i+1]]  
    int[] aux = new int[N];  
    for(int ai:a)  
        aux[cumsum[ai]++] = ai;  
    a = aux;      
}

analysis

running time: linear in time and in space.
stable sorting: that's why we need the cumsum and aux array...

3. LSD Radix Sort

LSD = least significant digit (for string sorting)
(assume strings all of same length)
idea:

consider chars from right to left
sort using dth character as key (using key-indexed counting)

important: the key-indexed counting should be stable.

analysis

time: W * N (W=length of string)

correctness:
prop. LSD sorting works
pf. by induction on i
prove that: after pass i, strings are sorted by last i characters.

implementation

public static LSDsort(String[] a, int W){//W=fixed width  
    int R = 256;//for ASCII chars  
    int N = a.length;  
    String[] aux = new String[N];  
    for(int d=W-1;d>=0;d--){//W passes  
        //key-indexed counting at digit d  
        int count = new int[R+1];  
        for(String s:a)  
            count[s.charAt[d]+1]++;  
        for(int i=0;i<N;i++)  
            count[i+1] += count[i];//count = cumsum(count), range of each key  
        for(String s:a)  
            aux[s.charAt[d]++]=s;//put each key in right place  
        for(int i=0;i<n;i++)  
            a[i]=aux[i];  
    }  
}

if keys are binary numbers → break into bit characters then apply LSD.
if not fixed length → some fix..
sort 1 million 32-bit integers? (Google/Obama interview) ⇒ LSD string sort~

4. MSD Radix Sort

most-significant-digit first (from left to right)
Idea.

partition array into R pieces according to first character (the count[] array is the partition)
recursively sort each subarrays

variable length: end-of-string are treated as before any char

implementation

private static void sort(String[] a, String[] aux, int lo, int hi, int d){  
    if(hi<=lo) return;  
    //sort by dth character  
    int count = new int[R+2];// R+2 as we are taking account into the end-of-strings!!  
    for(int i=lo;i<=hi;i++) count[ a[i].charAt(d)+2 ]++;//charAt(end-of-string)=-1  
    for(int i=0;i<R;i++) count[i+1] += count[i]; //cumsum  
    for(int i=lo;i<=hi;i++) aux[count[a[i].charAt(d)+1]++] = a[i];  
    for(int i=lo;i<=hi;i++) a[i] = aux[i-lo];// attention: aux is filled from index 0 instead of lo  
    //recursively sort each subarray (R subarrays in total, ranges stored in count[])  
    for(int r=0;r<R;r++)  
        sort(a, aux,lo+count[r], lo+count[r+1]-1, d+1);  
}

analysis

can recycle aux[], but not count[].
too slow for small subarrays (if len(subarray)<<R)
huge nb of subarrays because of recursion

improvement ⇒ cutoff to insertion sort...

characteristics of MSD sort:
examines just enough chars to sort.
→ can be sublinear in N.

MSD vs. quicksort

disadvantages for MSD:

random access of memory (cache inefficient)
too many instructions in inner loop
extra space for count[]
extra space for aux[]

disadvantages for qsort:

NlgN nb of string compares
has to rescan many chars for keys with long prefix matches

5. 3-way Radix Quicksort

⇒ combine benefits of qsort and MSD.

idea: do 3-way partition by the dth character.

less overhead than R-way partitioning for MSD
do not re-examine chars equal to the partitioning char

Implementation

modification of the 3-way qsort.

private static sort(String[] a, int lo, int hi, int d){  
    if(hi<=lo) return;  
    int lt = lo, gt = hi, i=lo+1;// use 3 pointers: lt, i, gt  
    char pivot = a[lo].charAt(d);  
    while(i<=gt){//invariant: a[lo,lt)<pivot, a(gt,hi]>pivot, a[lt,i]=pivot  
        char c = a[i].charAt[d];  
        if(c<pivot) exch(a,lt++,i++);  
        else if(c>pivot) exch(a,i,gt--);  
        else: i++;  
    }//3-way partition  
    sort(a,lo,lt-1,d);  
    if(pivot>=0) sort(a,lt,gt,d+1);//pivot<0 means end-of-string  
    sort(a,gt+1,hi,d);  
}

analysis

wrt. qsort:

from NlgN string compares to NlgN char compares.
avoids re-comparing long common prefixes.

wrt. MSD:

short inner loop
cache friendly
in-place

6. Suffix Arrays

(some applications of suffix array)

keyword-in-context search

given N chars (N huge), preprocess it to enable fast substring search.
⇒ suffix sort

generate suffix array (linear time & space )
sort on the suffix ⇒ brings repeated suffixes together

longest repeated substring

brute force algorithm

try all i, j as starts of indices, then compute longest common prefix (LCP)
→ O(D*N2) where D is length of longest repeated substring.

⇒ use suffix array

sorting suffix array will bring repeated substrings together
java code:

//int lcp(String s1, String s2) defined  
public static lrs(String s){  
    int N = s.length;  
    String[] suffix = new String[N];  
    for(int i=0;i<N;i++) suffix[i] = s.substring(i);//construct suffix array  
    Arrays.sort(suffix);// sort suffix array ==> using 3-way radix sort   
    // one pass to get longest repeated substring (bigest lcp)  
    String lrs = "";  
    for(int i=0;i<N-1;i++){  
        int lcp = lcp(suffix[i], suffix[i+1]);  
        if(lcp>lrs.length) lrs = suffix[i].substring(0, lcp);  
    }  
    return lrs;  
}

lrs worst-case input: lrs very long (say N/2).
⇒ quadratic for lrs and for sorting.

→ improvement of lrs for worst-case performance:
Manber-Myers algo
sketch:

phase 0: sort suffix[] on 1st char
phase i: given suffix[] is sorted based on first 2^(i-1) chars → create suffix[] sorted on first 2^i chars.

each phase: double the nb of chars sorted on.
maintain an inverse[] array, to make comparisons constant time.
performance: NlgN

String sorting summery

can have linear sort: use chars as array index
sublinear sort: not all date need to be examined
3-way radix qsort is asymptotically optimal

[Algorithms II] Week 3-1 Maximum Flow

2015-11-17T00:00:00+01:00

1. Introduction to Maxflow

Min-cut pb

input: edge-weighted digraph G, each edge e has weight("capacity") c[e]>=0, a source vertex s, a target vertex t.
def. an st-cut (A,B) is a partition of vertices into 2 disjoint sets A and B, with s in set A and t in set B.
def. the capacity of a cut (A,B) is sum of capacities of edges going from A to B (not considering B to A).

⇒ min-cut pb: find the cut (A,B) with min-capacity.

Max-flow pb

same input: graph G, source s, target t
def. an st-flow is an assignment of values to edges f: e→f[e] such that:
- capacity constraint: 0<=f[e]<=c[e] for any e;
- local equilibrium: for any vertex v (other than s or t), inflow=outflow;
def. the value of a flow f is the inflow at t. (assume no ingoing edge to s or outgoing edge to t)

⇒ max-flow pb: find f with max value.

remark: max-flow and min-cut are dual problems.

2. Ford-Fulkerson Algorithm

def. given a flow f for a graph, an "augment path" is an undirected path form s to t, if there exist df>0 ("bottleneck capacity") such that:

for forward edges e: can augment flow by df (not full: f[e]+df<=c[e])
for backward edges: can decrease flow by df (not empty: f[e]-df>=0)

def. residual capacity
- for forward edge e, residual-cap = c[e]-f[e]
- for backward edge e, residual-cap = f[e]

⇒ an aug-path is a path where each edge has residual capacity >0.

blocking edges: full forward edge or empty backward edge.

→ idea: increase flow along augment paths.
[algo]

start: 0 flow: f[e]=0 for all e.

find an augment path (and the corresponding df) in graph, and change the flows along the path by +/-df.

loop until no augment path exists. (ie. all path s→t are blocked either by a full forward edge or an empty backward edge, ie. by an edge with 0 residual capacity)

FF is a gernel algorithm:

3. Maxflow-Mincut Theorem

def. for a cut (A,B), the net flow across the cut (netflow(A,B)) is the sum of flows from A to B minus flows from B to A.

[flow-value Lemma]
For any flow f and any cut (A,B)⇒ netflow(A,B) = value(f).
pf.
induction on the size of set B.
base case, when B={t}, by def we have netflow(A,B) = value(f)
when moving any vertex v from A to B:
* netflow(A, B) augment by flow(A→v)+flow(B→v)=inflow(v),
* netflow(A, B) decrease by flow(v→A)+flow(v→B)=outflow(v),
* by equilibrium of flow, netflow(A',B')=netflow(A,B)=value(f)

ex. (A: gray vertices, B: white vertices)

[cor] outflow(s)=inflow(t)=value(f)

[weak duality]
For any flow f and any cut (A,B), ⇒ value(f) <= capacity(A,B).

[Augmenting path Th]
A flow f is maxflow iff there is no augment path.

[maxflow-mincut Th]
value(maxflow) = capacity(mincut).

pf.
for any flow f, prove the equivalence of the 3 following statements:
i. there exists a cut st: capacity(cut) = value(f).
ii. f is a maxflow.
iii. there is no augmenting path wrt f.

[i⇒ii]

suppose cut(A,B) st: capacity(A,B)=value(f)
⇒ by weak duality, for any other flow f', vlaue(f')<=capacity(A,B)=value(f)

[ii⇒iii] (eqv to prove ~iii⇒~ii)

suppose there is an aug-path from s to t, of bottleneck capacity=df,
⇒ by improving f with df, we get a f' > f

[iii⇒i]

suppose there is no aug-path, ie, all path from s to t are blocked by some full-forward edge or empty backward edge.
⇒ let A:=vertices connected with s by a path with no blocking edges, and B := the rest
(so once we get a maxflow, we can compute the mincut in this way)

→ for all edges across A and B, all forward edges are full, all backward edges are empty
⇒ capacity(A,B) = netflow(A,B) = value(f) by flow-value lemma
CQFD... 过瘾...

4. Running Time Analysis

getting a mincut form maxflow? → easy (as discussed in the pf above)
computing an aug-path? → BFS
does FF algo always terminate? how many augmentations? → ...

integer capacity graphs

special case of FF algo: edge capacities are integers between 1 and U.

invariant: flow is always integer all along FF algo.

[prop] nb of augmentations <= value of maxflow.
pf. each augmentation will add flow by >=1.

[integrality Th] There exist an integer-valued maxflow.

Bad case for FF

nb of augmentation == value of maxflow
(each time, the path through the middle edge is chosen as aug-path)

can be easily avoided⇒ by using shortest(nb of edges)/fastest(biggest df) path

Performance of FF depends on the algo for choosing aug-path:

5. Java Implementation

representation of flow graph

flow edge:

each e= v→w, have flow f[e] and capacity c[e].

flow graph:

put e in both v and w's adj-list.

flow augmentation (by delta)
- for forward edge e, f[e] += delta
- for backward edge e, f[e] -= delta

Residual graph Gr

def. For a flow f and a graph G, the residual graph Gr is obtained by:

for each edge e=v→w, (with c[e] and f[e]) in G, put in Gr:
e1=v→w, with weight=c[e]-f[e]
e2=w→v, with weight=f[e] (即两个方向上的weight都为residual capacity)

(rmq: Gr is just a weighted digraph, not a flow graph)

[prop] Augment path in G is equivalent to a path in Gr (df of aug-path in G = min edge weight in Gr).
(但是实现的时候其实不用显式构造Gr, 只需BFS的时候修改一下即可)

APIs

这里的API设计的非常合理... 导致每一部分的代码量都不大... NB

flow-edge:
rmq. both calculate residual-cap and augmentation need to specify a direction, so we need a index v as parameter for these 2 functions.

public class FlowEdge{   
    private final int v, w;   
    private final double capacity;   
    private double flow=0.0;   
    FlowEdge(int v, int w, double cap);   
    int from();   
    int to();   
    int other(int v);   
    double capacity();   
    double flow();   
    double residualCapTo(int v);// residual capacity   
    void addFlowTo(int v, double delta);// augment residual flow   
}

flow graph:

public class FlowNetwork{   
    private Bag<FlowEdge>[] adj;//use adj-list representation for flow graph   
    FlowNetwork(int V);   
    void addEdge(FlowEdge e);   
    Iterable<FlowEdge> adj(int v);// both incoming and outgoing edges   
    ...   
}

FF algo:
- use a function hasAugPath() to test termination
- use a function bottleNeck() to get delta
- if a augpath is found, use two arrays reached[] and edgeTo[] to get the augpath (find the path backwards).

code:

public class FordFulkerson{   
private boolean[] reached; //reached[v] indicates if a path s-->v exists in Gr, used in DFS   
private FlowEdge[] edgeTo;// edgeTo[v] = last edge on the path s-->v   
private double value=0.0;// value of flow   
public FordFulkerson(FlowNetwork G, int s, int t){   
    while(this.hasAugPath(G,s,t)){   
        double delta = this.bottleNeck();   
        for(int v=t; v!=s; v=edgeTo[v].other(v))   
            edgeTo[v].addFlowTo(v, delta);   
        this.value += delta;// each time the flow value augments by delta   
    }   
}   
private double bottleNeck(){//bottleneck-cap = min residual flow on the aut-path   
    double bottleneck = 9999999;   
    assert(reached[t]);// the aug-path should exsit   
    for(int v=t; v!=s; v = edgeTo[v].other(v))   
        bottleneck = Math.min(bottleneck, edgeTo[v].);   
    return bottleneck;   
}   
private boolean hasAugPath(FlowNetwork G, int s, int t){   
    // perform a BFS    
    Queue<Integer> q = new LinkedList<Integer>();   
    this.reached = new boolean[G.V()];   
    this.edgeTo = new FlowEdge[G.V()];   
    q.add(s);   
    while(!q.isEmpty()){   
        int v = q.deque();   
        for(FlowEdge e:G.adj(v)){   
            int w = e.other(v);   
            if(!reached[w] && e.residualCapTo(w)>0){// modified BFS: valid edges are those with  residualCap>0   
                edgeTo[w] = e;   
                reached[w] = true;   
                if(w==t) return true;// t is reached by BFS  
                q.enqueue(w);   
            }   
        }   
    }// BFS while loop    
    return false;   
}   
}//class FF

6. Maxflow Applications

关键是建模很巧妙...

ex1. bipartite matching pb

二分图的最大匹配问题. (有点像marriage stable问题...但是不一样因为没有preference order)

⇒ is there a way to match all students to a job?
ie. given a bipartite graph, find a perfect matching.

modeling

add source s and target t

all edges from s to students: capacity=1

all edges from companies to t: capacity=1

all edges from student to company: capacity=INF

⇒ find maxflow in the graph

when no perfect matching: mincut can explain why

in the above case, student 2,4,5 can only be matched to 7,10
⇒ mincut can help us find such cases!

recall: how to get mincut from maxflow

mincut = (A,B), where:
A:=vertices connected with s by a path with non blocking edges,
B := the rest
(blocking edges: full forward edge or empty backward edge on path)

ex.

let S=students on s side of mincut (in above case, S={2,4,5})
let T=companies on s side of mincut (in above case, T={7,10})
|S|>|T|, that's why no perfect matching!

ex2. baseball elimination

(前三列是目前成绩, 后面四列是接下来赛程矩阵)
Montreal is mathematically eliminated → easy to see
→ Philly is mathematically eliminated also !

another case:

Detroit is mathematically eliminated !

whether team-4 still has a chance to win?
modelling

remaining games flow from s to t.
use team-pairs ans teams as vertices
carefully chosen capacities(see below)

⇒ team 4 could win iff all flow from s are full (ie. all match points can be repartitioned over other teams without depassing team 4's maximum wins).
总之很巧妙....

[Algorithms II] Week 2-2 Shortest Paths

2015-11-15T00:00:00+01:00

1. Shortest Paths APIs

context: directe, weighted graphs.

shortest path variants

in terms of vertices:

source-sink: form one vertex to another
single source: from one vertex to all others (considered in this lecture)
all pairs

constraints on edge weights:

nonnegative weights
arbitary weights
eculidean

cycles:

no directed cycles
no negative cycles

APIs

for weighted directed edge:

public class DirectedEdge{   
    DirectedEdge(int v, int w, double weight);   
    int from();   
    int to();   
    double weight();   
}

for edge-weighted digraph:

public class EdgeWeightedDigraph{   
    private final Bag<DirectedEdge>[] adj;    
    EdgeWeightedDigraph(int V);   
    void addEdge(DirectedEdge e);   
    Iterable<DirectedEdge> adj(int v);   
    int V();   
}

for single source shortest-path:

public class SP{   
    SP(EdgeWeightedDigraph G, int s);//s is the source node   
    double distTo(int v);//dist from s to v   
    Iterable<DirectedEdge> pathTo(int v);// shortest path from s to v   
}

2. Shortest path properties

goal: single-source shortest path
prop. a shortest-path-tree (SPT) exists.
这个结论以前没见过... 不太evident吧... 没有太想明白

⇒ consequence: can represent this SPT by 2 vertex-indexted arrays:

double distTo[v] = shortest path length s → v
DirectedEdge edgeTo[v] = last edge to v in shortest path, edgeTo[s] = null

private int distTo[] = new int[V];   
private DirectedEdge edgeTo[] = new DirectedEdge[V];   
public double distTo(int v){   
    return this.distTo[v];   
}   
public Iterable<DirectedEdge> pathTo(int v){   
    Stack<DirectedEdge> path = new Stack<DirectedEdge>();   
    for(DirectedEdge e = this.edgeTo[v]; e!=null; e = edgeTo[e.from])   
        path.push(e);   
    return path;   
}

Edge relaxation

dynamic prog:

distTo[v] = length of known shortest path from s to v
distTo[w] = length of known shortest path from s to w
edgeTo[w] = last edge in the known shortest path form s to w

(consider edges one by one)
def. edge e = v-->w relaxes if e.weight+distTo[v] < distTo[w]. → update distTo[w] and edgeTo[w].

private void relax(DirectedEdge e){   
    int v = e.from(), w = e.to();   
    if(distTo[w]>distTo[v]+e.weight()){   
        distTo[w] = distTo[v]+e.weight();   
        edgeTo[w] = e;   
    }   
}

optimality conditions

prop. optimality conditions
distTo[] is the solution iff:

distTo[s] = 0

distTo[v] is the weight of some path from s to v

for any edge e = v → w, distTo[w] <= distTo[v]+e.weight()

Generic algo

just relax all edges...

prop. the above generic algo gives the SPT from s.

implementations of the algo:

Dijkstra (nonnegative edge)
Topological sort (!) (no directed cycles)
Bellman-Ford (no negative directed cycles)

3. Dijkstra's Algorithm

(non-negative edges)

consider vertices in increasing distance from s.*

add vertex to the SPT, relax all edges from that vertex.

each time: take the closest vertex to s that is not in the SPT (ie. whose dist is not determined yet), add the vertex, and relax all its outgoing edges.

prop. Dijkstra works.
pf.
each edge e = v → w is considered exactly once (when vertex v is added to SPT). distTo[w] <= distTo[v]+e.weight() after relaxing e
ineq holds until algo terminates
* distTo[w] decrease monotonely,
* distTo[v] will not change because each time we choose distTo[v] smallest, and all edges non-negative.
* CQFD by optimality condition.

implementation

Use an IndexMinPQ to store vertices.

public class DijkstraSP{   
    private int dist[];   
    private DirectedEdge edgeTo[];   
    private IndexMinPQ<Integer,Double> pq;   
    DijkstraSP(EdgeWeightedDigraph G, int s){   
        this.dist[] = new int[G.V()];   
        this.edgeTo[] = new DirectedEdge[G.V()];   
        this.pq = new IndexMinPQ<Integer,Double>(G.V());   
        for(int v = 0; v<G.V(); v++)  dist[v] = 999999;   
        dist[s] = 0;           
        pq.insert(s, 0.0);   
        while(!pq.isEmpty()){   
            int v = pq.delMin();   
            for(DirectedEdge e:G.adj(v))    
                relax(e);// decrease key or insert vertices to pq   
        }   
    }//constructor()

    private relax(DirectedEdge e){   
        int v = e.from(), w = e.to();   
        if(dist[w]<dist[v]+e.weight()){   
            dist[w] = dist[v]+e.weight();   
            edgeTo[w] = e;   
            if(pq.contains(w)) pq.decreaseKey(w, dist[w]);   
            else pq.insert(w, dist[w]);   
        }   
    }//relax()   
}//class

Dijkstra is in fact a Prim algorithm ! Both are algos that compute a spanning tree.

in Prim: each time takes the vertex closest to the tree (and is for undirected graph).
in Dijkstra: each time takes the vertex closest to source s (and is for directed graph).

DFS BFS are also computing spanning tree!
...NB!!

complexity

V insertions to PQ (each vertex is added to the SPT)
V delMin from PQ
E decrease key

→ depends on the PQ implementation.
in our implementation, O(ElogV)

4. Edge-Weighted DAGs

in a DAG, it's easier to find the shortest path ?
⇒ yes!
simple algo:

consider vertices in topological order

relax outgoing edges form this vertex.

correctness

prop.

topo-sort algo computes SPT in any DAG (even with negative weights) in time O(E+V) (linear time!).

pf.
each edge is relaxed exactely once. distTo[w] <= distTo[v]+e.weight() after relaxing e
ineq holds until the algo terminates because:
* distTo[w] decrease monotonely,
* distTo[v] will not change because of topo-order, no edge pointing to v after v is relaxed.
* CQFD

implementation

public class AsyclicSP{   
    AsyclicSP(EdgeWeightedDigraph G, int s){   
        // init dist[] and edgeTo[]...   
        Topological topo = new Topological(G);   
        for(int v: topo.order())   
            for(DirectedEdge e:G.adj(v))   
                relax(e);   
    }   
}

application

seam-carving

Resizing images non-uniformly without distortion.

好神奇!
Grid DAG of pixels, edges are pointed to 3 downward neighbors.
横向缩小: 删掉一个"seam"

longest path in DAG → just negate all edge weights, as this algo is OK for negative edges.
parallel job scheduling

感觉有点运筹的意思, 关键是建立一个DAG, 比较有技巧性:
- add source and sink
- split each job to 2 vertices (begin and end)

⇒ use longest path to schedule jobs

5. Negative Weights

只是给所有边增加weight并不能解决负权边问题.

⇒ need a different algo.

prop. SPT exists iff there is no negative cycles.

Bellman-Ford

一句话总结Bellman算法: 每次relax所有的边, 进行V次.

(因为relax k次以后, s到v的最短路径的长度小于等于k.)

prop. Bellman-Ford works, complexity is O(EV).
pf.
after i passes, found shortest path contain at most i edges.

code:

public class BellmanFordSP{   
    BellmanFordSP(EdgeWeightedDigraph G, int s){   
        // init dist[] and edgeTo[]...   
        for(int i=0;i<G.V();i++)   
            for(DirectedEdge e:G.edges())   
                relax(e);   
    }

}

improvement

if dist[v] is not changed during one pasee → no need to relax incoming edges any more.
→ much faster in practice.

FIFO implementation: maintain a queue of vertices whoses dist is changed.

Finding negative cycle

add two method to SP:

boolean hasNegCycle();   
Iterable<DirectedEdge> negCycle();

prop.

if a vertex u is updated in phase V, there exists a negative cycle, and can track edgeTo[u] to find the cycle.

原因是最短路径的长度小于V, 如果大于V则说明存在negative cycle.

negative cycle application

arbitrage detection

建模: graph of currencies, edge weight = exchange weight (complete graph).
⇒ find a cycle whose product of edge is >1.
⇒ take logs to make a shortest-path pb.
⇒ take minus log, then try to find a negative cycle.

NB!!

Summery

[Algorithms II] Week 2-1 Minimum Spanning Trees

2015-11-11T00:00:00+01:00

1. Introduction to MSTs

Given: undirected connecte graph G with positive edge weights.
def. Spanning tree T
is a subgraph of G, that is both tree (connected, acyclic) and spanning(all vertices are included).

⇒ Goal: find a spanning tree with minimum weight sum.

2. Greedy Algorithm

assumptions for simplification:

edge weights are distinct
graph is connected

→ MST uniquely exists.

cut property

def. a cut of a graph is a partition of its vertices into 2 non-empty sets.
def. a crossing-edge (wrt a cut) is an edge connecting vertex from one set to another.

prop. Given any cut, the crossing edges with minimum weight is in the MST.

proof.
Given a cut. {S1,S2} are the two set of vertices, let e be the min-weighted edge among all crossing-edges.

If e is not in the MST
→ exist another crossing-edge, f, in the MST (otherwise not connected)
→ adding e to the MST will create a cycle (tree property)
→ the edge f will be in this cycle
→ removing f and adding e will give us another spanning tree (!)
→ this new spanning tree has smaller weight sum ⇒ contradiction, CQFD.

Greedy MST algo

[algo ] Greedy MST

initialize: all edges not selected (colored gray)

find any cut with all crossing-edge gray

use this cut and select the min-weighted crossing edge (color the edge as black)

repeat V-1 times.

prop. the greedy algorithm gets the MST.
pf.

any selected (black) edges are in the MST (according to the cut property)
If we haven't selected V-1 edges → there is always a cut with all crossing-edges gray. (证明algo不会卡死)

(if edge weight not distinct, the proof fails, but can be fixed)

efficient implementations:

how to choose the cut each time?
how to find min-weighted crossing-edge?

⇒ Kruskal & Prim

3. Edge-Weighted Graph API

Edge API

→ Edge abstraction: make Edge comparable.

public class Edge implements Comparable<Edge>{   
    Edge(int v, int w, double weight);   
    int either();// get one of the endpoint of edge (as we are in undirected graph contex here)   
    int other(int v);// get the other endpoint    
    int compareTo(Edge that);// compare by edge weight   
    double weight();   
}

Edge-weighted Graph API

adj-list implementation: Bag<Edge>[] adj;(for undirected graph, each edge appears twice in adj)

public class EdgeWeightedGraph{   
    private final int V;   
    private final Bag<Edge>[] adj;   
    EdgeWeightedGraph(int V){   
        this.V = V;   
        this.adj = (Bag<Edge>)new Bag[V];   
        for(int v=0;v<V;v++) adj[v] = new Bag<Edge>();   
    }   
    void addEdge(Edge e){// use the Edge class instead of directly v and w   
        int v = e.either(), w = e.other();   
        adj[v].add(e);   
        adj[w].add(e);   
    }   
    Iterable<Edge> adj(int v){//get Edges incident to v   
        return adj[v];   
    }   
    Iterable<Edge> edges();// get all Edges   
}

(allow self-loops and parallel edges)

MST API

public class MST{   
    MST(EdgeWeightedGraph G);//compute the MST   
    Iterable<Edge> edges();// selected edges in the MST   
    double weight();// sum of all edge weights in MST   
}

4. Kruskal's Algorithm

[algo]

consider edges in ascending order of weight,

add the edge to MST unless it creates a cycle.

In the running of Kruskal: we have several small connect components and they merge with each other until we get MST.

correctness

prop. Kruskal's algo works.
pf
(idea: proove that Kruskal is a special case of the greedy algorithm, ie. how to select the specific cut)
suppose Kruskal's algo selects(colored black) an edge e=v-w
→ select a cut = vertices connected to v in the (constructing) MST; and the rest vertices.
→ for this cut, there is no black crossing edges
→ moreover among all crossing edges of the cut the edge e has the smallest weight!! (by def of Kruskal) CQFD

implementation

how to test if adding an edge will create a cycle ?

DFS from v to w? → O(V)
⇒ Union-Find ! O(lg*V) ☺ (almost constant time)

if find(v)==find(w), then we know adding e will create a cycle.

considering edges in order? → use a prority queue.

public class KruskalMST extends MST{   
    private Bag<Edge> mst = new Bag<Edge>();   
    public KruskalMST(EdgeWeightedGraph G){   
        MinPQ<Edge> pq = new MinPQ<Edge>();   
        // build pq --> can be optimized to O(n) if build bottom-up   
        for(Edge e: G.edges()) pq.insert(e);   
        UF uf = new UF(G.V());// build a UF of V elements   
        while(!pq.isEmpty() && mst.size()<G.V()-1){   
            Edge e = pq.delMin();   
            int v = e.either(),w=e.other(v);   
            if( uf.connecte(v,w) ) continue;   
            uf.union(v,w);   
            this.mst.add(e);           
        }   
    }   
    public Iterable<Edge> edges(){   
        return this.mst;   
    }   
}

complexity

running time: O(ElogE)

5. Prim's Algorithm

since 1930...
Idea: start from a vertex and grows the tree T to MST.

[algo]

Add to the tree T the edge that have exactely one endpoint in T and with minimum weight,

repeat V-1 times.

In the running of Prim: there is always ONE connnected component .

Correctness

prop. Prim's algo works.
pf.
suppose edge e is the min-weighted edge connect a vertex in T with a vertex out of T.
→ select the cut = vertices in the tree T; vertices out of T
→ by def, there is no black crossing edge
→ e is the min-weighed edge by def of Prim. CQFD

implementation

challenge: how to find such an edge (connect T and other vertex, with min weight) ?
⇒ priority queue

"lazy" implementation

[algo]

Maintain a PQ of edges that connect T and the rest vertices.

e = pq.delMin(), e = v-w, if v and w are both in T (as edges in pq might become obsolete as T grows) ⇒ just disregard it

to maintain the pq: add all incident edges(with other endpoint not in T) of the newly added vertex to pq

public class LazyPrimMST{   
    private Bag<Edge> mst;   
    LazyPrimMST(EdgeWeightedGraph G){   
        boolean[] marked = new boolean[G.V()]; // vertices in T   
        MinPQ<Edge> pq = new MinPQ<Edge>();    
        this.mst = new Bag<Edge>();   
        marked[0] = 0; // add vertex 0 to T   
        for(Edge e:G.adj(0))    
            pq.insert(e);// add edges to pq   
        while(!pq.isEmpty() && this.mst.size()<G.V()-1){   
            e = pq.delMin();   
            int v = e.either(), w = e.other(v);   
            if(marked[v] && marked[w]) continue;//ignore obsolete edges   
            v = marked[v] ? w : v;// v is the newly added vertex   
            marked[v] = true;   
            for(Edge e:G.adj(v)){   
                if(!marked[e.other(v)])   
                    pq.insert(e);   
            }   
        }   
    }   
}

Running time: O(ElgE)

space: O(E) in worst time.

"eager" implementation

Idea:

use a PQ of vertices, priority of vertex v := min-weight of edge that connects v to T.

[algo]

Get from pq the vertex v that is closest to T, add it to T.
Update pq -- consider v's incident edge e=v-w:

if w in T → ignore

else:

if w in pq → add w to pq

else → if v-w has smaller weight than the current priority, update w's priority.

repeat till get V-1 edges.

key implementation component: a MinPQ that supports priority(key) update.

class IndexMinPQ<Key extends Comparable<Key>>{   
    IndexMinPQ(int N);// indices of elements: 0...N-1   
    void insert(int i, Key key);   
    void decreaseKey(int i, Key key);// update the key(priority) of element-i   
    int delMin();   
    int size();   
}

implementation of such a PQ:

Use same code as standart PQ (maintain a heap[] array).
Elements are always accessed by "index", in range 0...N-1. maintain 3 parallel arrays: keys[], pq[], qp[]:

keys[i]: is the priority of element i (the element with index=i)

pq[i]: is the index of the element in the heap position i (ie. in heap[i] is pq[i]th element )

qp[i]: is heap position of element i ( ⇔ the ith element is in heap[qp[i]] )

to decreaseKey(i,key): change keys[i], then call siftup(qp[i])

summery of pq implementations:

6. MST Context

unsolved pb: does a linear MST algo exists?

(recap: for UF, tarjan has prooved that linear algo doesn't exist — although Nlg*N is fast enough...)
@_@...

(这个Yao是清华那个Yao吧?)

Euclidean MST

Given N points in plane, edge weight := Euclidean distance. (dense graph, E = V2)
→ exploit geomerty, O(NlgN)

clustering

k-clustering (~ dist-fcn)
single-link clustering (def. dist of clusters = dist of 2 closest elements in each cluster)
→ Kruskal...

[Algorithms II] Week 1-2 Directed Graphs

2015-11-09T00:00:00+01:00

1. Intro to digraphs

Has profound differences wrt undirected graphs.

def: digraph
edges: have directions
vertex: distinguish indeg and outdeg

digraph pbs:

path/shortest path
topological sort: Can you draw a digraph so that all edges point upwards?
strong connectivity: Is there a directed path between all pairs of vertices?
transit closure
PageRank

2. Digraph API

public class Digraph{   
    Digraph(int V);   
    void addEdge(int v, int w);// edge is directed   
    Iterable<Interger> adj(int v);// vertices reached by outgoing edges   
    int V();   
    Digraph reverse();// <--new methode wrt undirected graph   
}

representation: adj-list, ie. an array of bags.
Bag<Integer>[] adj;// prec vertices

3. Digraph Search

BFS and DFS can be applied to digraphs.

reachability

find all vertices reachable from vertex-s.
use the same DFS as for undirected graphs.
→ application: programme control-flow analyse, garbage collection.

DFS is the basis for a lot of digraph pbs: 2-satisfiability, Euler path, strongly connected component.
multiple source shortest path:

⇒ use DFS but enque all vertices in the set.
→ application: web crawler(DFS not suitable for crawling)

4. Topological Sort

application. precedence schedule, java compiler (cycled inheritance), ...

def. topo-order
is a permutation of vertices, where for each vertice v→w, w is behind v in the permutation.

def. DAG
directed acyclic graph.
prop. for a digraph, topological order exists iff graph is a DAG.

algo: ⇒ use DFS~
reverse DFS postorder

def. postorder
is the order of the vertices that we have finished (ie. we have visited all reachable vertices from this vertex).

implementation

这个以前的blog写过...

private boolean[] visited;   
private Stack<Integer> revPostorder;// stores the vertices in reverse post order   
private void dfs(Digraph G, int v){   
    visited[v] = true;   
    for(int w: G.adj(v))   
        if(!visited[w])   
            dfs(G, w);   
    //** now we know the vertex v is "finished" **   
    revPostorder.push(v);   
}   
public Iterable<Integer> topoOrder(Digraph G){   
    for(int v=0;v<G.V();v++)   
        if(!visited(v)) dfs(G,v);// visit all cc   
    return revPostorder;   
}

proof

prop. reverse post-order of a DAG is in topological order.
(这个证明蛮精彩)
pf.
for any edge v→w, when dfs(v) is called:

case 1: dfs(w) is called and returned, so w is done before v in post-order;
case 2: dfs(w) is not called, it will be (in)directly get called by dfs(v), so dfs(w) finishes before dfs(v);
case 3: dfs(w) is called but NOT returned (ie, w not finished) → exist path from w to v ⇒ graph is not a DAG! (cycle detection)

5. Strong Components

For undirected graphs: connected components can be solved with dfs or UF.

def. Strongly-connected
v and w are strongly-connected if exist path from v to w and w to v.
→ is an equivalent relation.

def. Strong Component
subset of V where each pair are strongly-connected.

Goal: compute all strong components(scc) in a digraph.

linear time DFS solution: Tarjan (1972)

(developed version: a two-pass linear-time algorithm)

Intuition: scc for G is the same for G.reverse().

Kernel DAG: contract each scc into a single vertex.

Idea:

compute topological-order in the kernel DAG.
run DFS, consider vertices in reverse-topo-order

[Algo]
1. compute topo-order in G.reverse (just a DFS in the reversed graph)
2. run DFS in original G, visit unmarked vertices in topo-order of G.reverse. (instead of visiting vertices by their index)

⇒ each time we finish a dfs from a vertex, we get a scc!
太精彩了!!!

proof: tricky, cf book...(貌似Werner课上讲过..)

implementation

private int[] scc = new int[V]; // scc[v] is the index of the SCC that v belongs to   
private int sccCount = 0;   
private boolean[] visited = new boolean[V];   
public getSCC(Digraph G){   
    // 1. get topo-order in reverse graph   
    Iterable<Integer> topoOrderGR = topoOrder(G.reverse());   
    // 2. run dfs in original graph, run on vertices using the above topo-order    
    for(int v:topoOrderGR)// <-- only difference from the standard topo-order algo   
        if(!visited[v])   
            dfs(G, v, sccCount++);//increment sccCount everytime we done a component   
}   
private dfs(Digraph G, int v){   
    // run dfs from v, and all touched vertices are marked in sccId's SCC   
    visited[v] = true;   
    scc[v] = sccCount;   
    for(int w:G.adj(v))   
        if(!visited[w]){   
            scc[w] = sccCount;   
            dfs(G,w);   
        }   
}

[Algorithms II] Week 1-1 Undirected Graphs

2015-11-08T00:00:00+01:00

1. Intro to graphs

Graph: vertices connected by edges.

terminology:

path: sequence of vertices connected by edges
cycle: path with same starting and ending vertex
two vertices are connected: if there is a path between

ex of graph problems:

path: or connectivity
shortest path
cycle
Euler tour (ouii..)
Hamilton tour
MST
bi-connectivity: is there a vertex whose removal disconnects the graph?
planarity
isomorphism

2. Graph API

graph representation

vertex representation: use integers between 0 and V-1

anormalies: self-loop and multiple edges are possible

public class Graph{   
    Graph(int V);   
    void addEdge(int v, int w);   
    Iterable<Integer> adj(int v);   
    int V();// nb of vertices   
    int E();// nb of edges   
}

print all edges:

basic functions:

static int degree(Graph g, int v){   
    int deg = 0;   
    for(int w:G.adj(v)) deg++;   
    return deg;   
}   
static int nbOfSelfloops(Graph g){   
    int cnt = 0;   
    for(int v=0; v<G.V(); v++)   
        for(int w:G.adj(v))    
            if (w==v) cnt++;   
    return cnt/2;   
}

edge representation

set-of-edge implementation: a list of all edges ⇒ can lead to inefficient implementation
adj-matrix implementation: maintain a 2d (V*V) boolean array ⇒ space complexity too heavy
adj-list implementation: vertex-indexed array, each array entry is a Bag (类似桶bucket) ⇒ sutable for sparse graphs

adj-list implementation:

private final int V;   
private Bag<Integer>[] adj;   
public Graph(int V){   
    this.V = V;   
    this.adj = (Bag<Integer>[]) new Bag[V];// java cannot create generic array   
    for(int v = 0; v<V; v++) adj[v] = new Bag<Integer>();   
}   
public addEdge(int v, int w){   
    adj[v].add(w);   
    adj[w].add(v);// if undirected graph   
}

3. Depth-First Search

Tremaux maze exploration: trace back when no unvisited vertices availiable.
动画好看...
DFS goal: systematically search through a graph.
design pattern: decouple graph data and graph processing.

public class Paths{   
    Path(Graph G, int s);// graph G and source s   
    boolean hasPathTo(int v);   
    Iterable<Integer> pathTo(int v);   
}

algo:

注意每次访问节点以前就将其mark.

implementation

用一个boolean数组visited[]作为标记
为了找到一条具体的路径(ie, 一系列节点), 维护一个prev[]数组, 存放当前节点是从哪个节点走过来的.

//public class DFSpaths extends Paths...   
boolean[] visited = new boolean[V];   
int[] prev = new int[V];   
public void dfs(int v){   
    visited[v] = true;   
    for(int w: G.adj())   
        if(!visited[w]) {   
            prev[w]=v;   
            dfs(w);   
        }   
}   
public Iterable<Integer> pathTo(int v){   
    Stack<Integer> s = new Stack<Integer>();   
    for(int x = v; x!=s; x = prev[x])    
        s.push(x);   
    return s;   
}

properties

prop.
DFS visite all edges in time propotional to the sum of their degrees(ie. nb of edges).

4. Breadth-First Search

not recursive algo.
maintain a queue, add to queue for all vertices not-marked.

implementation

use visited[] to mark vertices
use a prev[] array to get explicit path

use a dist[] array to record the shortest dist from v to source (can use dist to replace visited)

public void bfs(Graph G, int s){   
    boolean visited[] = new boolean[G.V()];   
    int prev[] = new int[G.V()];   
    int dist[] = new int[G.V()];   
    Queue<Integer> q = new Queue<Integer>();   
    visited[s] = true;   
    q.push(s);   
    while(!q.isEmpty()){   
        int v = q.dequeue();   
        for(int w:G.adj(v))   
            if(!visited[w]) {   
                prev[w] = v;   
                visited[w] = true;   
                q.enqueue(w);   
            }      
    }   
}

property

prop.
BFS computes the shortest path from s to all vertices using time propotional to E+V.

intuition: BFS examines nodes by increasing distance

5. Connected Components

dealing with connectivity(equivalence) queries ⇒ answer in constant time (with preprocessing).

public class CC{   
    boolean connected(int v, int w);   
    int count();// nb of CCs   
    int id(int v);//id for a CC   
}

⇒ Union-Find ? ⇒ Use DFS!!

def. connected component is a maximal set of connected vertices.

algo: for each unmarked vertex, run dfs(with increasing cc id)...
after the preprocessing, we can get the array id[] and cc count cnt...

6. Graph Challenges

some typical pbs
pb1. bipartite graph

Can we divide vertices into 2 subsets, where all edge go from one subset to other.
⇒ can be done with dfs. cf. booksite

pb2. cycle detection
⇒ simple using dfs.

pb3. Euler cycle
Find a cycle that uses all edges exactely once.
[Euler] a graph is Eulerian iff all vertices have even degree.
⇒ typical diligent algo students can do. cf. booksite

pb4. Hamilton cycle
Find cycle that visits each vertex exactly once.
⇒ intractable (typical NP-complete pb)

pb5. isomorphism of graphs
Are two graphs identical except for vertex names?
⇒ no one knows...

pb6. planary graph
Lay out a graph in the plane without crossing edges?
⇒ expert level. exists linear time algo based on DFS by Tarjan, but too complicated.

最短路径三剑客: Floyd, Dijkstra, Bellman

2015-10-18T17:00:00+02:00

weighted graph的最短路径问题有三个非常有名的算法, 分别以三个大牛的名字命名, 今天用尽量简洁的篇幅一一介绍.

简单起见(这回只写伪代码好了), 对于图的定义如下:

node index = {1,2,...,n}
e[u,v] = distance of edge(u,v); if (u,v) is not an edge, e[u,v]=INF
令N=点的数量, M=边的数量

任意两点最短路径: Floyd-Warshall

Floyd算法解决的问题是: 对于任意两个节点s(source)和t(target), 求s到达t的最短路径.

Floyd算法的思想是动态规划:

定义d(i,j,k)为点i到点j之间, 只允许借道节点1...k的最短路径

初始化: d(i,j,0)=e[i,j](即i到j之间不经由其他任何中转节点的最短路径)

更新dij的公式就是: d(i,j,k)=min( d(i,j,k-1), d(i,k-1,k-1)+e[k,j])

更新n次

每次更新dij的意思就是: 现在从i到j可以经过节点k了, 那么看一下ij之间从k这个点经过的话(i → k → j 这条路)能不能缩短dij.

i到j最短路径最终就是: d(i,j,n) 即i到j的路程可以经过1~n中的任何中转节点.

伪代码特别短(其实真代码也一样短....):

for all [i,j]: // initialize   
    d[i,j] = e[i,j]    
for k = 1 ~ n: // relax for k times:    
    for all [i,j]:   
        d[i,j] = min(d[i,j], d[i,k] + e[k,j])

核心代码只有最后三行... 运行结束后d[i,j]就保存着任意i和j之间的最短路径长度.
程序主循环n次, 每次要处理遍历所有的ij组合, 所以复杂度是O(N^3).

单源最短路径: Dijkstra

Dijkstra算法解决的问题是: 没有负权边的情况下, 从源节点s到其他任意节点t的路径长度.

维护一个dist数组, dist[i]就表示(目前为止)s到i的最短距离.
对于每个元素, 标记是否其dist是否已经确定不再更改(或者说维护两个集合: 一个集合的dist确定, 另一个未确定).

Dijkstra算法是一种贪心策略: 每次在未确定最短路径的节点里挑选距离s最近的那个点, 把这个点标记为已经确定dist, 然后对从这个点出发的边进行松弛.

为了标记每个点, 这里用一个bool数组表示: determined[i]为true表示i的dist已经是最短路程, 为false表示还不确定.

算法如下:

初始化dist[i] = e[s,i], determined[i]全为false

在dist未确定的元素里(determined[i]==false)寻找一个dist最小的节点u:

标记u的dist已经确定determined[i]=true

用u的所有出边进行松弛: s到i如果经过(u,i)这条边会不会变近? dist[i] = min(dist[i], dist[u]+e[u,i])

重复循环直到所有的点都确定dist(or 重复N遍即可: 每次只会确定一个新的节点的距离)

伪代码:

for i in 1~n:   
    dist[i] = e[s,i]   
    determined[i] = false   
loop N times:    
    u = argmin(dist[i]) among all i that determined[i]==false   
    determined[u] = true // determine one node at each loop     
    for v such that e[u,v]<INF:   
        dist[v] = min( dist[v], dist[u]+e[u,v] )

以上代码的复杂度为O(N^2), 不过如果用堆来优化寻找最近的u的距离, 复杂度可以变得更低.

有负权边的单源最短路径: Bellman-Ford

Dijkstra算法的缺点在于不能处理边长为负数的情况, 而这就是Bellman-Ford算法解决的.

Bellman算法也是一种动态规划(动态规划这个东西就是Bellman提出来的):

定义d(i,k)为源点s到i最多经过k条边的最短距离

初始化: d(i,1)=e[s,i]

每次更新di的公式: for all (u,v): d(v,k)=min( d(v,k-1), d(u,k-1)+e[u,v] )

更新n-1次

为什么是更新n-1次? 因为s到i的最短路径至多只有n-1条边(即s到i的路径经过了所有n个点).
注意每次更新, 需要把所有的边试一遍: 看看用每条边(u,v)能不能松弛dv.

伪代码:

for i in 1~n:   
    dist[i] = e[s,i]   
loop n-1 times:    
    for all (u,v) such that e[u,v]<INF:   
        dist[v] = min( dist[v], dist[u]+e[u,v] )

核心代码也是最后三行... 太tm精妙了!!

外层循环N-1次, 内层循环M次, 所以代码的复杂度是O(NM).

以下问题有空再写...

negative cycle
A*

深度优先搜索(DFS)小结

2015-10-18T00:00:00+02:00

今天总结一下也许是搜索问题里最重要的算法: DFS !

由于树可以看成是一个graph, 这里还是只写对于graph的DFS算法. Graph类的定义还是用每一个节点保存邻居信息:

public class GraphNode{      
    int val;      
    List<GraphNode> neighbors;      
}

为了防止重复, 仍然用一个HaseSet记录走过的节点:

HasheSet<GraphNode> visited = new HasheSet<GraphNode>();

Recursive DFS

首先写递归版本的DFS, DFS就是一条路走到底, 不撞南墙不回头, 所以递归写起来很自然: 每到一个节点, 标记其已经访问过了, 然后对于邻居里面没有访问的节点继续递归进行DFS.

递归的DFS代码非常简洁:

public void DFS(GraphNode nd){      
    System.out.println(nd.val);    
    visited.add(nd);   
    for(GraphNode next: nd.neighbors){   
        if( !visited.contains(next) )   
            DFS(next);   
    }   
}

虽然这个算法很短, 但是它非常重要, 回溯算法(backtracking)其实就相当于在问题的求解域做一个dfs. 另外拓扑排序也是基于递归dfs进行一点点修改.

Non-recursive DFS

非递归版本的dfs同样很重要, 因为毕竟非递归的版本效率高一些, 另外这个算法和bfs非常相似, 只不过把队列queue换成了栈stack而已:

public void DFS(GraphNode start){      
    Stack<GraphNode> s = new Stack<GraphNode>();   
    q.push(start);      
    visited.add(start);      
    while(!s.empty()){      
        GraphNode cur = s.pop();      
        System.out.println(cur.val);      
        for(GraphNode next: cur.children){       
            if(!visited.contains(next)){      
                s.push(next);      
                visited.add(next); // mark node as visited when adding to stack!       
            }      
        }      
    }//while      
}

同样要注意的一点就是在把一个节点入栈的时刻就将其标记为已访问.

(for Trees) DFS with depth

和上次bfs一样, 对于树而言, 在dfs搜索的过程中也可以记录该节点所在的depth.

非递归版本的程序就是用一个和上面s平行的栈记录深度, 程序和"BFS with distance"很像. 递归版本只要在函数签名里加上一个depth的参数即可. 这两个实现都很简单, 就不写了.

注意这个只对于树有意义, 对一个图而言没有depth一说...

DFS for binary tree: Preorder traversal

dfs另一个有用的性质是: 对于二叉树而言, dfs得到的节点顺序正是其前序遍历(preorder traversal)的顺序.

其实前序遍历的定义就相当于是一个递归版本的dfs了:
[preorder(node)] = node.val + [preorder(node.left)] + [preorder(node.right)]

DFS with path

如果在访问到某一个节点的时候想同时获得到该点的路径, 其实也不麻烦.

对于递归版本的dfs而言, 可以在参数里面用一个List记录到当前节点的路径.

非递归的版本的话... 貌似不是很trival, 需要对stack做好维护, 可能需要一个hashmap什么的... 以后有空了再写写.

Cycle Detection

判断一个有向图是否存在回路是一个非常重要的问题, 简单修改dfs就可以做到了.

在递归版本的dfs里, 我们对每一个点改为三种标记:
未访问过(0), 正在访问其邻居节点(1), 已经访问完该节点以及所有该节点可以到达的节点(2).

什么时候会出现回路呢? 就是当前节点v的一个邻居u的状态为1的时候. 因为该节点状态为1, 即还没有把它以后的节点全部遍历, 所以当前节点v肯定可以从u到达, 而现在又可以从v到达u, 所以构成一个回路.

为了表示一个节点的三种状态, 我们把visited的定义改一下, 定义为一个hashmap:
HasheMap<GraphNode, Boolean> visited = new HasheMap<GraphNode, Boolean>();

节点不在visited表示还未访问过, 节点对应为false表示正在访问, 节点对应为true表示已经访问该节点以及所有可以从它到达的节点.

写一下代码:

public void DFS(GraphNode nd){      
    visited.put(nd, false); // mark as status-1   
    for(GraphNode next: nd.neighbors){   
        if( !visited.contains(next) )   
            DFS(next);   
        else if(visited.get(next)==false) // found cycle   
            System.out.println("Cycle detected!!!");   
    }// now all touchable nodes from nd are visited   
    visited.put(nd, true); // mark as status-2   
}

非递归版本的话貌似也不很容易, 暂时不写了, 以后有空了考虑一下...

Topology Sort

这一节(以及上一节)参考这个非常棒的视频: https://class.coursera.org/algo-003/lecture/52

拓扑排序是一个dfs的应用, 所谓拓扑排序是指在一个DAG(有向无回路图)里给每个节点定义一个顺序(v1...vn), 使得按照这个顺序遍历的节点, 每一个节点vi都是之前遍历过的的节点(v1 ~ vi-1)所指向的(或没有任何其他节点指向的).

好像还没说清楚... 拓扑排序的一个应用就是对于各种依赖性(比如学习课程A需要先学习过课程B)组成的图寻找一个节点遍历的顺序使其可行.

propositions:

拓扑排序的结果不唯一.

有回路的图不存在拓扑顺序.

如果一个节点没有出边, 那么它可以放在拓扑排序的最后面(没有节点以来它).

如果一个节点没有入边, 那么它可以放在拓扑排序的最后面.

简单修改一下递归的dfs就可以处理拓扑排序: 维护一个计数器K(初始化为n=所有节点数), 每当一个点已经遍历完毕(所有通过这个点可以到达的点都已经被走过)以后, 就把这个点的顺序设为K, 同时减少K.

就用一个HashMap来为每个节点关联一个序号好了:
HasheMap<GraphNode, Integer> order = new HasheMap<GraphNode, Integer>();

public void DFS(GraphNode nd){      
    for(GraphNode next: nd.neighbors){   
        if( !visited.contains(next) )   
            DFS(next);   
    }// all touchable nodes from nd are visited   
    order.put(nd, K--);   
}

是不是特别简单, 太神奇了!

上面只是对于一个点进行的, 为了给所有点拓扑排序, 只要从一个没有出边的节点出发进行遍历, 一直运行到所有的节点都已经访问过为止.

广度优先搜索(BFS)小结

2015-10-08T00:00:00+02:00

今天总结一下广度优先搜索(BFS). BFS是树/图的遍历的常用算法之一, 对于没有边权重的图来说可以计算最短路径.
由于树的BFS只是图的BFS的一种特殊情况, 而且比较简单不需要visited标记, 这里只写一下图的BFS好了.
先定义一个Graph类, 这里在每一个节点保存邻居信息:

public class GraphNode{   
    int val;   
    List<GraphNode> neighbors;   
}

BFS for trees/graphs

图的遍历需要注意不走重复节点, 所以需要一个HashSet(名字叫visited)来保存哪些节点已经访问过了. 需要注意的是, 在把一个节点放进队列queue的时刻就要把它放进visited, 而不是在队列里取出来的时刻再放.

public void BFS(GraphNode start){   
    LinkedList<GraphNode> q = new LinkedList<GraphNode>();   
    HasheSet<GraphNode> visited = new HasheSet<GraphNode>();   
    q.push(start);   
    visited.add(start);   
    while(!q.empty()){   
        GraphNode cur = q.poll();   
        System.out.println(cur.val);   
        for(GraphNode next: cur.children){    
            if(!visited.contains(next)){   
                q.push(next);   
                visited.add(next); // mark node as visited when adding to queue!    
            }   
        }   
    }//while   
}

BFS with distance

在BFS的同时我们可以记录从start节点到当前node的距离, 方法是把一个距离信息同时入队(封装一个Pair<GraphNode, Integer>), 或者使用一个与queue平行的队列保存距离信息.
在上面的代码中, 加入:

//...   
LinkedList<Integer> distq = new LinkedList<Integer>();   
distq.push(0);// distance from start to start   
//...   
// in the while(!q.empty()) loop:    
    int d = distq.poll();//get distance from start to current node   
    for(GraphNode next: node.children){   
        distq.push(d+1);// distance from start to next node   
        //...

对于Tree的情况来说, 这里的dist其实就是当前节点的深度depth.

properties

性质1:
每个节点node的distance都是node距离起始点start的最短距离.

性质2:
距离start近的节点(depth浅的节点)一定比距离start远的节点早被访问到.

这是对一个树BFS的时候节点的访问顺序:

BFS "by layer"

参考上面的性质, 可以一次处理"一层"的节点, "一层"的意思是指所有节点距离start的距离相同. 代码在while循环里不是一次poll一个节点, 而是一次把queue的内容处理完, 然后换新的queue进入下一次while循环. 代码重新写一下:

public void BFS(GraphNode start){   
    ArrayList<GraphNode> q = new ArrayList<Tree>();   
    HasheSet<GraphNode> visited = new HasheSet<GraphNode>();   
    q.push(start);   
    visited.add(start);   
    while(!q.empty()){   
        ArrayList<GraphNode> newq = new ArrayList<Tree>();// create a new queue   
        for(GraphNode cur: q){// deal with all nodes in the queue    
            System.out.print(cur.val+", ");// all nodes in q are of the same distance/depth   
            for(GraphNode next: cur.children)    
                if(!visited.contains(next))   
                    { newq.push(next);visited.add(next); }   
        }   
        System.out.println();   
        q = newq;//replace q with newq   
    }//while   
}

以上程序每次打印一行, 第i行包括了距start距离为i的所有节点.
由于这样的话每次不必在队首poll出元素(而是依次处理所有queue的元素), 所以可以改用ArrayList. 此时while循环里的不变量是: 所有q里面的节点距离start的距离都相同.

complexity

假设一个图有N个节点和M条边, BFS会走遍所有节点, 时间是O(N), 然后由于每个节点会检查所有的出边, 最终所有的边都会被检查过, 时间是O(M), 所以BFS的时间复杂度是O(N+M).

队列里面最多可能存放所有节点, 空间复杂度为O(N).

Mindmap of algorithms & data structures

2015-10-05T00:00:00+02:00

Here is a mindmap of the common algorithms and data structures, it can give an overview of the algorithmic terms.

I shall update its content later on. And maybe write some blog entries on some of the items.

This mindmap is drawn using xmind.

[Algorithms I] Week 6 Hash Tables

2015-09-01T00:00:00+02:00

Can we do better than BST if we do not need ordered operations ?

(No compare methods, use equals method)

Idea: save items in an array.
Hash function: method for calclulating the array index of a key.

Issues:

computing hash function
equality tests
collision resolution

Classic space-time tradeoff.

1. Hash Functions

Goal: scramble the keys uniformly to produce a table index.

effcient to compute
all indices are equally likely for any key

challenge: need different approach for different key types.

Java's hashing

All java object has a methode int hashCode()
requirement:

if x.equals(y) ⇒ x.hashCode()==y.hashCode()
(hopefully) if x.equals(y)==false ⇒ x.hashCode!=y.hashCode()
Default implementation: memory address for x.
Custom implementations for standard types: Integer, Double, String, File, URL, Date...

Strings: Horner's method — compute a polynome.

recipe for user-defined types:

Modeular hasing

(hash code VS hash function)

Hash code: 32-bit integer between -2^31 and 2^31-1 (can be negative!!)
Hash function: integer between 0 and M-1 (used directly as array index, should >=0)

BUG code:

private int hash(Key k){   
    return k.hashCode()%M;   
}

→ bug: number returned can be negative !

1-in-billion bug code:
return Math.abs(k.hashCode()) % M;

→ bug: Math.abs() returns negative numbers for -2^31

correct code:
just take the first 31 bits: x&0x7fffffff
return k.hashCode()&0x7fffffff % M;

另一个hash的思路是用random(用hashcode作为随机种子)

uniform hasing assumption

each key is equally likely to hash to an integer between 0 and M-1

一些数学结论:

2. Separate Chaining

one strategy for collision resolution.

idea: Using a table of size M < N, build a list for each of the table positions.

implementation

public class SeparateChainingHashST<Key, Value>{   
    private int M = 97;   
    private Node[] st = new Node[97];   
    private static class Node{   
        Object key, val; // no generic array creation!    
        Node next;   
        public Node(Key k, Value v, Node nxt){...}   
    }   
    int hash(Key k){   
        return (k.hashCode()&0x7fffffff) % M;   
    }   
    public Value get(Key k){   
        int h = hash(k);   
        for(Node x=st[h]; x!=null; x=x.next)   
            if(k.equals(x.val)) return (Value)x.val; //cast   
        return null;   
    }   
    public void put(Key k, Value v){   
        int h = hash(k);   
        for(Node x=st[h]; x!=null; x=x.next)   
            if(k.equals(x.val))    
                {x.val = v; return;}   
        st[h] = new Node(k,v,st[h]);   
    }   
}

注意内部类Node里用的是Object — 因为不能声明generic array.

analysis

proposition
Under the assumption of uniform hashing, the number of keys in a list is within a constant factor to N/M.
proof.
binominal distribution.

M times faster than sequential search
→ typical choice: M ~ N/5

3. Linear Probing

Another approch for collision resolution.

idea: open addressing
Use an array of size M>N, when a key collides, find next open slot.

insert: when occupied, move pointer until got open slot
search: when not found, move pointer until find or reach open slot
delete: set key to null, then for all items behind this key: set to null, then insert this key......

array M must be larger than N ⇒ array resizing is necessary.

implementation

public class LinearProbingHashST<Key, Value>{   
    private int M, N;   
    private Object[] keys;//cannot use generic array   
    private Object[] vals;   
    public void put(Key k, Value v){   
        int i = hash(k);   
        for(;keys[i]!=null;i=(i+1)%M)   
            if(keys[i].equals(k))    
                {vals[i]=v; return;}   
        keys[i]=k;   
        vals[i]=v;   
    }   
    public Value get(Key k){   
        int h=hash(k);   
        for(;keys[h]!=null;h=(h+1)%M)   
            if(k.equals(keys[h])) return (Value)vals[h];//ugly cast   
        return null;   
    }   
}

cluster

def. cluster
A contiguous block of items.

New keys are more likely to hash into a cluster.

Knuth parking pb:

(当年Knuth就是在证明了这个以后决定写那套书)

Typical choice: N/M ~ 1/2 (3/2 for search hit and 5/2 for search miss)

4. Hash Table Context

widely used in applicataions.

cost for computing hash VS cost for searching:
hashCode() for strings in java 1.1: exemain only 8-9 evenly spaced characters to save time.

uniform hashing assumption: performance not guaranteed...

one-way hash functions
Hard to find a key that hash to a desired value, or 2 keys having the same hash value.

used for fingurprints or store passwords, but too expensive for ST implementations.

Seperate chaining VS linear probing

variations

ST choice: hash tables VS balanced search trees

[Algorithms I] Week 5-2 Geometric Applications of BSTs

2015-08-29T00:00:00+02:00

1. 1d Range Search

Goal: intersections of geometric objects.

Solution: BST

1d range search

operations required:

insert
search
delete
range search: all keys between k1 and k2
range count: how many keys are between k1 and k2

→ find points on an interval

implementation by BST

range count
using the rank() function for the BST (or use the size of a tree)

注意什么时候要加1...

public int size(Key hi, Key lo){   
    if(contains(hi)) return rank(hi)-rank(lo)+1;   
    else return rank(hi) - rank(lo);   
}

range search
类似inorder traversal的方式:

find in left subtree (if could fall into range)
check current node
find in right subtree

running time: R+lgN (R=nb of nodes in range)

2. Line Segment Intersection

Orthognal line segment intersection search:
find all intersections given N horizontal/vertical lines

Non-degeneracy assumption: all x-coord and y-coord are distinct.

naive algo: check all pairs...

Sweep-line algorithm

sweep a vertical line from left to right.
- when hit the left end of horizontal-segment (h-seg) → insert into a BST
- when hit the right end of a h-seg → delete from BST
- when hit a vertical-seg: ⇒ 1d range search !

关于怎么sweep的:
没有仔细讲, 不过我觉得就是把所有的x坐标排好序, 有个skyline问题也是涉及如何sweep的.

proposition
running time is NlgN+R (R=nb of intersections).
proof.

Sort by x-coord (or use PQ) → NlgN
insert/delete y-coord to BST → NlgN
range search → NlgN + R

3. Kd-trees

An extension of BST: 2d-keys.

insert: insert 2d points
search
range search: find all keys lying in a 2d rectangle (h-v rectangle)
range count

gird implementation

divide space into a M-by-M grid (uniform squares).
space: N + M^2
time: 1 + N/M^2
→ choose square to balance space and time.
problem: points are not uniformly distributed.

2d tree

Use a tree to represent the subdivision of the space.

2d tree: recursively divide the space into 2 halfplanes

construct the 2d tree by adding points: alternating between horizontal and vertical partitioning for each level of tree.

Data structure: BST alternating x and y-coords as key.

Range search for 2d tree

find all points lying in a rectangle.
依然类似tree traversal算法:

check point in node
find in left subtree (if could be in range — the rectangle intersects the splitting line)
find in right subtree

analysis
Typical case: R + lgN
worst case: R+ sqrt(N) (even if tree is balanced)
(proof is hard)

Nearest Neighbour seach

find closest point to a query point.

check dist from query point to node
check in left tree (if could contain a closer point — 和两点连线与splitting line的角度有关系)
check in right tree

analysis
typical case: lgN
worst case: N

Flocking boids
3 simple rules to get a simulation of flocking.

Kd tree

partition the k-dim space into 2 halfspaces.
cycle through k dimensions.

(居然时一个本科生发现的!)

Nbody simulation:
treat clusters as an aggregated node

4. Interval search tree

1d interval search:　data are intervals

insert interval
search interval
delete interval
intersection query: find all intervals that intersects (lo,hi)

Nondegeneracy assumption: all left endpoint of intervals are distinct.

API:

put(Key lo, Key hi, Value val)   
get(Key lo, Key hi)   
delete(Key lo, Key hi)   
Iterable<Key> intersects(Key lo, Key hi)

Interval search tree:

BST using left endpoint as key
in each node: store the max right endpoint of the subtree

insert
类似BST, 加上维护一下maxendpoint即可.

search
search any one interval that intersects (lo,hi)

if node intersects, return
if left.maxendpoint < lo: go right
else: go left

*proof. *
主要证明一点: if no intersection to left ⇒ then no intersection to the right

5. Rectangle intersection

Goal: find all intersection among N rectangles.
(non degeneracy assumption: all x and y are distinct)

bottom line: linearithmic algo.

sweep-line algorithm:
sweep vertical line from left to right.

when hit left part of a rect ⇒ put into an interval search tree
when hit right part of a rect ⇒ remove interval
every time befor adding ⇒ check intersection

reduces the 2d rect intersection pb to 1d interval search pb.

complexity:
NlgN+RlgN

summery:

[Algorithms I] Week 5-1 Balanced Search Trees

2015-08-26T16:00:00+02:00

goal: lgN for insert/search/delete operations (not necessarily binary trees..)
3 algo: 2-3 tree, (left leaning) red-black tree, B-tree

1. 2-3 Search Trees

def. 2-3 tree

allow 1 or 2 keys per node, & 2 or 3 children per node:
- 2-node: one key, 2 children (ordinary BST node)
- 3-node: 2 keys, 3 children (3 children: less, between, more)
perfect balance: every path from root to null link has the same length (2-3 tree的一个超好的性质, 类似于一个满二叉树!)
symmetric order: inorder traversal gives ascending order (和BST类似)

search
Just follow the correct link... Natural generalization of search in BST...

insert

case 1: insert into a 2-node at bottom

just convert a 2-node into a 3-node

case 2: insert into a 3-node at bottom
- create a temporary 4-node (three keys)
- move middle key in 4-node into parent, split the rest two keys into two 2-nodes

if parent becom a 3-node → continue the process
if arrived at the root (root is a 4-node with three keys): split it into three 2-nodes

splitting a 4-node: can be done in constant time (local transformation).

Analysis

Invariant: maintains symmetric order and perfect balance.
proof.
each transformation maintains the order and the balance, all possible transformations:
这个图很好, 3-node的插入一共有三种情况: 自身是root/父亲是2-node/父亲是3-node

performance
every path from root to null link has the same length.

Implementation

direct implementation is complicated:
bottom line: Could do it, but there's a better way.

2. Red-Black BST

LLRB tree: left-leaning red-black tree.

BST representation of the 2-3 trees
use internal left-leaning links for 3 nodes

红色link即为internal left leaning link (红黑树就是这么来的), 用红色link连接起来的组成一个(虚拟的)3-node 或4-node.

3-node用一个red link表示:

4-node用两个red link表示:

⇒ or or

example:

properties

no node has two red links (不可以一个节点连两个red link)
every path from path to null link has the same number of black links (想象所有red link都变为horizontal)
all red links lean left

representation

Each node has only one link from parent
⇒ add a boolean to encode color of links (the color of the link from parent).

private class Node{   
    private Key key;   
    private Value val;   
    Node left, right;   
    boolean color;//true means red     
}    
private boolean isRed(Node nd){   
    if (nd==null) return false;   
    return nd.color;   
}

insert to parent 操作: 只需把color变为RED即表示该节点被变成了和父节点一起的一个(虚拟)节点.

elementary operations

left-rotation
(def: convert a right-learning red link to left. )

(symmetric ordering and perfect black balance are maintained)

private Node rotateLeft(Node h){    
    Node s = h.right;   
    h.right = s.left;   
    s.left = h;   
    s.color = h.color;   // not = BLACK   
    h.color = RED;   
    return s;   
}

right-rotation
(temporarily turn a left-leaning red link to right)

private Node rotateRight(Node h){...}

right rotation 是为了应对这种情况:
rotateRight(c) ⇒

color-flip
(split a 4-node, with three kyes — two red links)

private void filpColor(Node h){   
    h.color = RED;   
    h.left.color = BLACK;   
    h.right.color = BLACK;   
}

Implementation

Basic strategy
Maintain one-to-one correspondence with 2-3 tree by applying elementary operations.

search

Exactly the same as elementary BST. ( ⇒ The same code for floor and ceiling)

insert

Each insert will generate a red link (then should rotate to make it legal)

插入的时候有两种可能:

insert into a 2-node at the bottom

standart BST insert
if have red right link: rotateLeft

ex:

insert into a 3-node

有三种可能: insert into left/middle/right, right最简单, left捎复杂, middle最复杂, 见下图:

standard BST insert and color nodes
if necessary, rotate to balance 4-node, 比如:

flip colors to pass red link to upper level
if necessary, rotate to make all links left-leaning

ex:

ex2:

视频最后一段的demo太帅了! 叹为观止!!

Code

原来只有4种(其实是3种)情况要调整:

left = black, right = red

⇒ rotateLeft(a)

left =red, left.right = red [这个不会出现, 因为这对于下一层来说是case 1..]

⇒ rotateLeft(e) ⇒ 变为case 3

left = red, left.left = red

⇒ rotateRight(s) ⇒ 变为case 4

left = red. right = red

⇒ flipColor(r)

几个状态之间的转化:

**只要三行代码即可处理LLRB tree !! 老爷子牛逼...... **
(这个也是在2007年algo第四版的时候才刚刚弄出来的, 以前的代码要复杂)

private Node put(Node nd, Key k, Value v){   
    if(nd==null) return new Node(k,v,RED);   
    int cmp = k.compareTo(nd.key);   
    if(cmp==0) nd.val = v; // 这里不急着返回 -- same trick as for BSTs..   
    else if(cmp<0) nd.left = put(nd.left, k, v);   
    else nd.right = put(nd.right, k, v);   
    // modifications to maintain LLRB tree property:    
    if( isRed(nd.right) && !isRed(nd.left) ) nd = rotateLeft(nd);//case 1   
    //if( isRed(nd.left) && isRed(nd.left.right) ) nd.left = rotateLeft(nd.left);// case 2 -- never happen...   
    if( isRed(nd.left) && isRed(nd.left.right) ) nd = rotateRight(nd);// case 3   
    if( isRed(nd.left) && isRed(nd.right) ) flipColor(nd);//case 4   
    return nd;   
}

这三行代码越看越精妙......

Analysis

worst case: the left path is alternating red and black.
⇒ longest path <= 2 * shortest path (height<= 2lgN)

practical applications: height ~ 1.0 lgN

summery:

3. B-trees

setting: data access in file system.
Probe is much expensive than accessing data within a page.

Goal: access data using a minimum number of probes.

B-tree

def.
external nodes: contain just keys, not links
internal nodes: contain key-link pairs

def. B-tree
Generalize 2-3 trees by allowing up to M-1 keys per node:

= 2 keys in root
= M/2 keys in other nodes
external nodes contain client keys
internal nodes contain copies of keys to guide search

Searching

similar to BST/2-3tree
ex.

(Choose M as large as possible so that M links fit into a page)

Insertion

similar to 2-3 tree

Analysis

System implementations

system implementations of RBtree.
java:
java.util.TreeMap, java.util.TreeSet.

八卦1:

八卦2: Sedgewick 的朋友, Philippe Flajolet, 是一个X!

[Algorithms I] Week 4-2b Binary Search Trees

2015-08-26T00:00:00+02:00

(BST是锻炼递归代码的好题目)

1. Binary Search Trees

def. BST
A binary tree where each node has a key:
for every node, the key is larger than all nodes in left subtree, smaller than all nodes in right subtree.

Fields: key, val, left, right

Implementation

An inner class of BST nodes:

private class Node{   
    private Key key;   
    private Value val;   
    private Node left, right;   
    public Node(Key k, Value v){...}   
}

skeleton implementation of BST:

public class BST<Key implements Comparable<Key>, Value>{   
private Node root;   
private class Node{...}   
public Value get(Key k){...}   
public void put(Key k, Value v){}   
public void delete(Key k){}   
public Iterable<Key> iterator(){}   
}

search
recursive version:
(或者把这个函数写到Node类里面也可以. )

private Value get(Node nd, Key k){   
    if(nd==null) return null; // search miss   
    int cmp = k.compareTo(nd.key);   
    if(cmp==0) return nd.val; // search hit   
    else if (cmp>0) return get(nd.right, k);   
    else return get(nd.left, k);   
}

non-recursive version:

public Value get(Key k){   
    Node nd=root;   
    while(root!=null){   
        int cmp = k.compareTo(nd.key);   
        if (cmp==0) return nd.val;   
        else if(cmp>0) nd = nd.right;   
        else nd = nd.left;   
    }   
    return null;   
}

insert
recursive version:
(注意这个recursive函数的返回值不是void! 这里是一个trick: 返回的是在分叉以前的那个节点)

private Node put(Node nd, Key k, Value v){   
    if(nd==null) return new Node(k, v);   
    int cmp = k.compareTo(nd.key);   
    if(cmp==0) nd.val = v;   
    else if(cmp>0) nd.right = put(nd.right, k, v);   
    else nd.left = put(nd.left, k, v);   
    return nd;   
}

non-recursive version:
不如递归版本优美...

public void put(Key k, Value v){   
    Node nd = root;   
    while(true){   
        int cmp = k.compareTo(nd.key);   
        if(cmp==0) {   
            nd.val = v; break;   
        }   
        else if(cmp>0){    
            if(nd.right!=null) nd = nd.right;   
            else {nd.right = new Node(k,v); break;}   
        }   
        else if (nd.left!=null) {    
            if(nd.left!=null) nd = nd.left;   
            else {nd.left = new Node(k,v); break;}   
        }   
    }   
}

Analysis

complexity: depth of the BST.

shape of BST: depends on how the keys come in (order of insertion).

if keys come in random order: could be pretty well balanced.

BST and quick-sort partitionning
The root of BST is just the pivot in quick sort partitioning *
if all keys are distinct ⇒ one-to-one correspondence between quick sort and BST.

⇒ proposition
if all keys are distinct and come in randome order, the average number of compares for a search/insert is ~2lnN (or 1.39lgN).
proof.*
证明见quicksort那里的数学推导...

proposition (Reed, 2003)
N distinct keys come in random order, average tree height = 4.300lnN

Worst-case:
The tree becomes just like a linked list: ~N for insertion and search

2. Oredered Operations in BST

task: ordered opeartions

min()/max(): min/max key
deleteMin()/deleteMax()
floor(Key k)/ceiling(Key k): largest key <=k / smallest key >=k
rank(Key k): nb of keys < key
select(int i): key with rank=i
Iterator<Key> keys(lo, hi): iterates through [lo, hi]

min/max

easy
min: left-most
max: right-most

floor/ceiling

a little more complexed...

floor (ceiling is similar)

if k==nd.key

return nd.val

if k<nd.key

the floor must be in the left subtree

if k>nd.key

如果min(nd.right) > k: 返回nd.val

如果min(nd.right) <= k: go to right

public Value floor(Node nd, Key k){// largest element with key <= k       
int cmp = k.compareTo(nd.key);         
if(cmp==0) return nd.val;//case 1   
else if(cmp<0) return floor(nd.left, k);//case 2   
if (nd.right==null || min(nd.right).compareTo(k)>0) //case 3   
    return nd.val;   
else return floor(nd.right);   
}

他提供的版本和我写的不一样: 递归函数floor返回的也是一个Node:

rank/select

In each node, store the number of nodes in the subtree: add an extra field.

size

private class Node{   
    private int count;   
    //...   
}    
public int size(){   
    return size(root);   
}   
public int size(Node nd){   
    if(nd==null) return 0;// this is why we do not put size() inside the class Node!   
    return nd.count;   
}   
public void put(Node nd, Key k, Value v){   
    //.....   
    nd.count = size(nd.left)+size(nd.right)+1;//maintain count for each node   
    return nd;   
}

rank
(return nb of keys < k)

if nd.key==k

return size(nd.left)

if nd.key>k

return rank(nd.left, k)

if nd.key<k

return size(nd.left)+1+rank(nd,right, k)

private int rank(Node nd, Key k){   
    if(nd==null) return 0;//remember null case   
    int cmp = k.compareTo(nd.key);   
    if(cmp==0) return size(nd.left)   
    else if (cmp<0) return rank(nd.left, k);   
    else return size(nd.left)+1+rank(nd.right,k);   
}

select() similar...

iteration

Inorder traversal 中序遍历

public Iterable<Key> keys(){   
    Queue<Key> q = new Queue<Key>();   
    inorder(root, q);   
    return q;   
}    
private void inorder(Node nd, Queue<Key> q){   
    if(nd==null) return;   
    inorder(nd.left);   
    q.enqueue(nd.key);   
    inorder(nd.right);   
}

property
inorder-traversal gives the keys in ascending order.
(proof by induction)

3. Deletions in BST

one final function to implement: delete(Key k), deleteMin(), deleteMax()
→ and remember to update the count field...

(感觉这篇文章其实就讲的很清楚了: http://www.algolist.net/Data_structures/Binary_search_tree/Removal 这个在递归函数里使用了parent这个参数)

lazy approch

put(k, null), and leave the key in the tree (tombstone)
→ not good if have large number of tombstons...

deleteMin/Max

go the the left-most node → replace it with its right node.
Recusive function with the returning-node trick:

private Node deleteMin(Node nd){   
    if(nd==null) return null; // this might not happen   
    if(nd.left==null) return nd.right;   
    else nd.left = deleteMin(nd.left);   
    nd.count = size(nd.left)+1+size(right);//remember to maintain the count field   
    return nd;   
}

这个递归的技巧又一次使用了.

Hibbard deletion

first find node with the key to delete, 3 cases:

0 children:

simply set parent link to null

1 child:

replace parent link with the child

2 children (most subtle)
- first replace node key with smallest key in right subtree
- remove the smallest key in right subtree

code of Hibbard deletion
Again (for the 3rd time) use the return-nd trick...

private Node delete(Node nd, Key k){   
    if(nd==null) return null;// search miss   
    int cmp = k.compareTo(nd.key);   
    if(cmp>0) nd.right = delete(nd.right, k);   
    else if(cmp<0) nd.left = delete(nd.left,k);   
    else{   
        //if nd is the node to delete   
        if(nd.left==null) return nd.right;   
        if(nd.right==null) return nd.left;   
        Key k2 = min(nd.right);   
        nd.key = k2;   
        nd.right = delete(nd.right, k2);   
    }   
    nd.count = size(nd.left)+1+size(nd.right);   
    return nd;   
}    
public void delete(Key k){   
    root = delete(root, k);   
}

感觉用了recursive return-nd 这个trick的实现很漂亮.... 比那篇博客里放一个参数进递归函数以及用auxroot的办法要好不少...

Analysis

problem: not symmetric
If random insert and delete for a while ⇒ tree become much less balanced ! Tree height tend to be sqrt(N).

summery
BST is much better in average case, but not guaranteed for worst case.

[Algorithms I] Week 4-2a Elementry Symbol Tables

2015-08-25T00:00:00+02:00

1. Symbol Table API

key-value pair abstraction

insert a value with a key
given a key, search for its value

Association array abstraction

Associate a value to a key — generalized array: a[key]=val.

public class ST<Key, Value>{   
    void put(Key k, Value v);//remove key if value=null   
    Value get(Key k);//return null if key is absent   
    void delete(Key k);   
    boolean contains(Key k);   
    boolean isEmpty();   
    int size();   
    Iterable<Key> keys();//better to return an ordered sequence of keys   
}

conventions:

values are not null
get() returns null if key not present
put() can overwrite older value

→ some one-line implementations:

contains: return get(k)!=null;
delete: put(k, null);

Assume keys to be comparable: class ST<Key implements Comparable<Key>, Value>— can thus use compareTo() method.
Else → we can only use the equals() method...
Be careful when implementing the equals method: 坑不少...

2. Elementary implementations

naive implementations

using unordered linked list

ListNode{key, value, next}

search: scan through all keys ~N
insert: scan through, if not found, add to front ~N

using ordered array

using 2 arrays: keys[] (sorted), vals[]
⇒ can improve performance by binary search

search operation
write a function rank() that returns the number of keys < k searched.
找不到的时候: 比k小的元素个数=lo (lo>hi, 可以想想当hi=lo以后是怎么移动的)

private int rank(Key k){   
    int lo=0, hi=keys.length-1;   
    while(hi>=lo){   
        int mid = lo + (hi-lo)/2;   
        int cmp = keys[mid].compareTo(keys[k]);   
        if(cmp==0) return mid;   
        else if(cmp>0) hi = mid-1;   
        else lo = mid+1;   
    }   
    return lo;   
}

Using rank() to implement the get() method:

public Value get(Key k){   
    int rk = rank(k);   
    if(rk<N && keys[rk].compareTo(k)==0) return vals[rk];   
    return null;   
}

insert operation
Like insertion sort, time complexity is ~N for each insert.

summery:

3. Ordered Opeartions

When keys are comparable ⇒ provide more functionalities in the API.
for example:

min()/max(): min/max key
deleteMin()/deleteMax()
floor(Key k)/ceiling(Key k): largest key <=k / smallest key >=k
rank(Key k): nb of keys < key
select(int i): key with rank=i
Iterator<Key> keys(lo, hi): iterates through [lo, hi]

[Algorithms I] Week 4-1 Priority Queue

2015-08-24T00:00:00+02:00

1. API and elementary implementations

Collection: data struct for inserting and deleting items (ex. stack and queue).
Priority queue: a special kind of collection — remove largest/smallest element.

API:

public class Max<Kye implements Comparable<Key>>{   
    public MaxPQ();    
    public void insert(Key k);   
    public Key delMax();    
    public boolean isEmpty();     
    public Key max();   
    int size();

}

PQ client example

find M largest elements from N items. (N is too huge to store)
⇒ 思路: 用 MinPQ, 当size>M时删掉最小元素 — 最后剩下的就是最大的M个元素了.

elementary(naive) implementations

unordered array implementation
insert ~1,
remove ~N

ordered array implementation
insert ~N — ~ insertion sort,
remove ~1

goal:
insert O(lgN) & remove max O(lgN)

2. Binary Heaps

binary heap: a special kind of complete binary tree.

def. "complete binary tree"
All level except the last level are full, all nodes in the last level are as far left as possible.
完全二叉树大概长这个样子:

property: a complete binary tree with N items has height = lgN.

def. binary heap
Binary heap is a complete binary tree that satisfies: each node >= any of its 2 children ("heap ordering").

property of binary heap:

max element is root
height is lgN
each node still makes a heap (use this property to construct heap from bottom up)

Array representation of binary heap

Use an array a[] to represent a complete binary tree: very easy to get children and parent.

index starting from 1 (root = a[1], do not use a[0])
for node at index i, its children are: i*2 and i*2+1
for node at index i (i>1), its parent is: i/2

Implementation

简言之就是:

内部成员变量:

public class MaxPQ<Key implements Comparable<Key>>{   
    private Key[] a;   
    private int N;   
}

siftup (swim)
if a node is larger than its parent: exchange with parent until heap ordering is restored.

private void swim(int k){   
    while(k>1 && less(a[k/2], a[k]){   
        exch(a,k,k/2);   
        k /= 2;   
    }   
}

⇒ use siftup for inserting elements
when inserting an element, first append it to the end of the array, then siftup the element to its right position.

public void insert(Key k){    
a[N++]=x;   
swim(N);   
}

siftdown (sink)
if a node is smaller than its children: exchange it with the bigger child. 用公司高层变动来类比很形象...

private void sink(int k){   
    while(k*2<=N){   
        int maxIndex = less(a[k],a[k*2])? k*2 : k;   
        if(k*2+1<=N && less(a[maxIndex],a[k*2+1]) maxIndex = 2*k+1;   
        if(maxIndex==k) break;   
        exch(a,k,maxIndex);   
        k = maxIndex;   
    }   
}

⇒ use siftdown for deleting max operation
First put a[N] to the position of root, then sink. (注意要防止loitering)

public Key delMax(){   
    Key ret = a[1];   
    a[1] = a[N];   
    a[N--] = null; // avoid loitering   
    sink(1);   
    return ret;   
}

Analysis

insert: ~lgN
remove max: ~lgN

improvements:

use a d-way heap instead of a 2-way heap (for ex. 3-way heap, the children of i are i3, i3+1, i*3+2, the parent of i is i/3)
Fibonacci heap: insert is ~1, remove max still ~lgN, (but too complicated to use in practice).

considerations:

use immutable keys to avoid client from changing the keys (final keyword in java)
underflow/overflow: throw exceptions for delMax() when empty, use resizing array for implementation.
expand the API with other operations: remove an arbitrary item, change the prority of an item (like handling an array)...

3. Heapsort

Basic idea:

create a max heap with all N entries ("heap-construction")
then repeatedly remove all N items ("sortdown")

Implementation

heap construction
Construct the heap using a bottom-up method: build the heaps with small size first.

one-node heaps (jus leaf node of heap): need not consider
larger heaps: children(subtrees) are already heap-order → just perform sink operation for root

⇒ sink elements at indices from N/2 to 1

for(int k=N/2; k>=1; k++)   
sink(k);

→ complexity is linear, see analysis below.

heap destruction **
now that we have a max-heap, to get the sorted array in-place, simply do:

while(N>1){   
exch(a,1, N--);   
sink(1);   
}

Analysis

proposition
Heap construction uses <2N compares and exchanges.
proof
一个有h层的heap, 高度为k的subtree有 2^(h-k)个, 每个subtree最多可以交换k次(从root交换到最底层)
→ 每一层最多有 k*2^(h-k) 次交换

(第一个等式可以用数学归纳法证...)
详细证明见http://algs4.cs.princeton.edu/24pq/ (Q20答案)

proposition
Heapsort use < 2NlgN compares and exchanges.

Heapsort is the first in-place sorting algorithm with NlgN worst-case performance.

⇒ heapsort is optimal for time and space, but:

inner loop is longer than quicksort
poor use of cache memory (too many references to cache — look far away from the array entry when going down through the tree)
not stable: because it does long-distance exchanges

Summary:

4. Event-driven simulation

Goal: simulating the motion of N particles in elastic collision (using a priority queue).

model

bouncing balls (without collision)

Ball class:

challenge: which objects to check? How to do it efficently ?

Time driven simulation

upate the position of balls every dt seconds
if overlap detected: figure how the velocity change and continue the simulation.

problem with time-driven simulation:

1/2*N^2 overlap check per time unit
if dt small: too much calculation
if dt too large: might miss collisions

Event driven simulation

change state only when something happens

particles move in straight line before collision
→ collision time can be predicted
maintain PQ for collisions, key=collision time

collision prediction
input: each particle has radius s, position (rx, ry), velocity (vx, vy).

(美国高中物理这么凶残?...)

Implementation

anyway, 这个是封装好的类:

注意, 使用了count记录一个particle到目前为止的碰撞次数.

initialization: quadratic

Fill PQ with all potential particle-particle collisions
Fill (the same) PQ with all potential particle-wall collisions

**Main loop **

get next event from PQ, event time = t
ignore if the event is invalidated
advance all particles to time t
update velocities
predict future collisions and insert to PQ

判断invalidate:
一个event构造(predict)时记录粒子构造时刻的碰撞次数(countA, countB),
然后在调用invalidate的时候, 如果粒子的碰撞次数发生了改变, 则返回false.
(详见他们的代码: http://algs4.cs.princeton.edu/61event/CollisionSystem.java.html)

Event class:

[Algorithms I] Week 3-2 Quicksort

2015-08-22T00:00:00+02:00

(maybe best algorithm for sorting.)

1. Quicksort

Idea:

shuffle the array
Partition the array into two subarrays to left and right of pivot (now pivot is in its final position)

no larger entry to the left of pivot
no smaller entry to the right of pivot

sort each subarray recursively

Implemetation

The partition process:
这个方法也比较巧妙.
Use 2 pointers i and j (个人觉得用hi, lo, pivot更好...) :

→ a[i]>=a[lo], a[j]<=a[lo] (注意是大于等于/小于等于)
⇒ exchange i and j
→ Scan until i and j cross (ie. j<=i)
⇒ finally exchange lo with j
函数的签名定义的好: 把lo到hi部分的数组分成两部分, 并返回分割点的index.

private static int partition(Comparable[] a, int lo, int hi){   
    int i=lo+1, j=hi;   
    while(i<j){   
        while( i<=hi && less(a[i],a[lo]) ) i++; //a[i]>=a[lo]   
        while( j>=lo && less(a[lo],a[j]) ) j--; //a[j]<=a[lo]   
        if(i<j) exch(a,i++,j--);   
    }   
    exch(a,lo,j); //exchange pivot with j   
    return j; //j in its final position   
}

这个函数其实并不好写对:

test for cross pointers is not trival (ex. edge case: the pivot is the smallest/largest entry in the range)
i<=hi is necessary !
for keys equal to a[lo]: better to stop at them

invariance:

**Quicksort: **
使用partition函数和辅助sort函数(recursive). 注意在整个流程开始以前先shuffle一下.

private static void sort(Comparable[] a, int lo, int hi){   
    if(hi<=lo) return;   
    int pivot = partition(a, lo, hi);   
    sort(a,lo,pivot-1);   
    sort(a,pivot+1,hi);   
    return;
}   
public static void sort(Comparable[] a){   
    StdRandom.shuffle(a); // don't forget to shuffle the array   
    sort(a,0,a.length-1);   
}

The randomness is preserved: the subarrays after partitionning is still randomly ordered.

Analysis

Performance: ~40% faster than mergesort.

Best case

compares = NlgN

(each partition will divide the array in half)

**Worst case **

compares = 1/2*N^2

N+(N-1)+...+1
if the array is already in order, each partition will have one subarray of length=0

**Average case **

proposition **
On average, for array with N distinct keys, the #compares = ~2NlnN, #exchanges = ~1/2NlnN.
Proof.*
C(N) := # compares for N entries
pivot 在N个数离的排名是uniform的
接下来的数学推到很漂亮(不过可能没啥用..)

(上面最后一行写错了... 是2NlnN...orz) random shuffle: probalistic guarantee against worst case.

Pitfalls **
implementations will get quadratic performance if array:

is sorted or reverse sorted
has many duplicates (even if randomized)

Staility **
Quicksort is NOT** stable.
partitionning can make long range exchanges

Practical improvements

cutoff to insertion sort for <10 items

→ ~20% improvement
Or we can leave the small subarrays unsorted and sort them at last using insertion sort

estimate median by sampling 3 items

→ 10% improvement

2. Selection

Goal: given un array, find the kth largest item.

Upper bound for this problem: NlgN (just sort the array)
for small k (ex k=1,2,3), the upper bound is N (one-pass/two-pass)
Lower bound is N: at least have to look at everything

Quick select

Algo proposed also by Hoare:

partition the array into two arrays left of pivot and right of pivot.
if pivot==k: return
continue the partition for just one of the subarrays

类似于二分查找的过程....
注意这里是不用递归的! 因为partition函数返回的直接就是pivot在整个数组里的位置!

Implementation

privater static int partition(Comparable[] a, int lo, int hi){...}

public static Comparable select(Comparable[] a, int k){   
    StdRandom.shuffle(a);   
    int lo=0,hi=a.length-1;   
    while(true){   
        int j = partition(a,lo,hi);   
        if(j<k) lo=j+1;   
        if(j==k) return a[j];   
        else hi=j-1;   
        if(hi<=lo) break;   
    }   
    return a[k];//这里不太理解为什么会在hi<lo的时候直接返回a[k]   
}

Analysis

Proposition **
Quick selection takes linear* time on average.
proof *
intuitively, each partition will ct the subarray size in half:
N+N/2+N/4+... = 2N
formal analysis 略...

worst case: quadratic (but very rare to happen)

Theoretical results

3. Duplicate keys

if array contains many duplicate keys.

huge array
small number of distinct keys

**for mergesort **
insensitive... always ~NlgN compares.

for quicksort **
Will get quadratic time if not stop on equal keys. (found in 1990s)
mistake: put all items equal to pivot *to just one side *
→ N^2 compares if all keys are equal from lo to hi.
correct: put all items equal to pivot in their final place.

3-way partitionning

(by Dijkstra)
partition the array into 3 parts:

Dijkstra's 3-way partition algo:
使用3个指针: lt指向中间部分的左边界, gt指向右边界; i指针从左向右扫描, 算法很subtle:

lt=lo, gt=hi, i=lo
if a[i]==v : i++
if a[i]<v: exch(i,lt), i++, lt++
if a[i]>v: exch(i,gt), gt--
repeat until i and gt cross (i>gt)

invariance:

[lo, lt) all < v
[lt, i) all == v
(gt, hi] all >v

Implementation: 3-way quick sort

不必再写partition函数, 直接在sort递归函数里面.

private static void sort(Comparable[] a, int lo, int hi){   
    if(hi<=lo) return; // 递归函数别忘了先写终止条件...   
    int lt=lo, gt = hi;   
    Comparable v = a[lo];   
    for(int i=lo;i<=gt;){ //不能写 i++   
        if( less(a[i],v) )   
            exch(a, i++, lt++);   
        else if ( less(v,a[i]) )   
            exch(a,i,gt--);   
        else // v==a[i]   
            i++;   
    }   
    sort(a, lo, lt-1);   
    sort(a, gt+1, hi);  
}

当N个数有很多重复的时候, lower bound可以变小于NlgN:

And Sedgewick proved that the 3-wy partition is propotional to the lower bound....

4. System Sorts

Arrays.sort() in java:

import java.util.Arrays;

quicksort for primitive arrays, mergesort for objects: java设计者认为如果用obj array表示空间不是问题...

Pb in java's system sort: killer input exsit (havn't shuffle)...

总结一下学过的5/6种排序:

[Algorithms I] Week 3-1 Mergesort

2015-08-21T00:00:00+02:00

Two classical sorting algorithms: mergesort, quicksort.

1. Mergesort

Divide and conquer: top 10 algorithms of the 20th century, invented by von Neumann.

Idea:

divide array into 2 halves
recursively sort each half
merge two sorted halves

Implementation

Merge:
Goal: a[lo] to a[mid] and a[mid+1] to a[hi] are sorted ⇒ get a[lo] to a[hi] sorted.
→use an auxiliary array to copy data: using 3 indices i,j,k.

private static void merge(Comparable[]a, Comparable[] aux, int hi, int mid, int lo ){   
for(int k=lo;k<=hi;k++) aux[k]=a[k];   
int i=lo, j=mid+1, k=lo;   
while(i<=mid && j<=hi){   
    if(less(aux[j],aux[i])) a[k++] = aux[j++];   
    else a[k++] = aux[i++];   
}   
while(i<=mid) a[k++] = aux[i++];   
while(j<=hi) a[k++] = aux[j++];   
}

note: we use if(less(aux[j],aux[i]))instead ofif(less(aux[j],aux[i])), because less(a,b) == true iff a<b (strict), and we want to make mergesort stable. See section (5) below.

Assertion
We can also add assertions: assert isSorted(a, lo, mid); assert isSorted(a,mid+1, hi);
Enable/disable assertion at runtime:
java -ea MyProgram //enable assertion
java -da MyProgram //disable assertion: default
Best practice:
use assertions to check interval invariants;
do NOT use assert for external argument checking !

**Mergesort **
为了mergesort需要写两个辅助函数: merge和sort(recursive):

public class MergeSort extends AbstractSort{   
    public static void sort(Comparable[] a){   
        Comparable[] aux = new Comparable[a.length];   
        sort(a,aux,0,a.length-1);   
    }   
    private static void sort(Comparable[] a, Comparable[] aux, int lo, int hi){   
        if(hi<=lo) return;    
        int mid = (hi+lo)/2;    
        sort(a,aux,mid+1,hi);   
        sort(a,aux,lo,mid);   
        merge(a,aux,hi,mid,lo);

    }   
    private static void merge(Comparable[]a, Comparable[] aux, int hi, int mid, int lo ){   
    // as before...   
    }   
}

有几点注意的:

注意递归终止条件是hi<=lo, 不是==.
实现的代码不难, 但是定义好辅助函数的参数并不简单... 比如在merge里使用了mid作为参数.
aux的数组直接写在了函数参数里面, 好处是可以防止反复声明数组带来的开销.
发现他们写的时候mid都是写成: mid = lo+(hi-lo)/2, 查了一下发现这样的原因是为了防止hi+lo整数溢出...嗯有道理.

Analysis

proposition (time)
Mergesort takes at most NlgN compares and 6NlgN array access.
Proof.
def: C(N)=#compares for N elements, A(N)=#array access for N elements, the recurrence eq:
C(N) <= 2C(N/2) + N; C(1) = 0 (最多N次比较: 每个a[]的数都由比较得到)
A(N) <= 2A(N/2) + 6N; A(1) = 0 (~~这里不太理解为什么是6N, 怎么数都是4N或者5N啊?......~~2N复制到aux, 2N复制回去, 2N比较)

(上图蓝色的一列是extra cost)
或者用递推公式, 发现D(N)/N的递推关系:

或者用数学归纳法:

Proposition(memory)
Mergesort takes N extra memory.
⇒ mergesort is NOT in-place sorting method.

def. "in-place"
A sorting algo is in-place if it uses <C*lgN extra memory.
ex. insertion sort, shellsort, selection sort...

Practical improvements

use insertion sort for small arrays:

cutoff to insertion sort for <7 items.

⇒ lead to 20% improvement!

stop if alread sorted: *a[mid]<a[mid+1]! *

switch the role of a[] and aux[]

2. Bottom-up Mergesort

bottom-up version of mergesort without recursion.
Idea:

pass and merge subarrays of size 1 in pairs
repeat for subarrays of size 2,4,8,....

implementation

看着上面那个图好写一点... 就是每次用更大的size来两两merge一遍数组

public static void sort(Comparable[] a){   
    Comparable[] aux = new Comparable[a.length];   
    for(int sz=1;sz<a.length;sz*=2){   
        for(int i=0;i<a.length-sz;i+=sz*2)   
            merge( a,aux,i,i+sz-1,Math.min(i+sz*2-1) );//<-注意mid和hi的计算方法   
    }   
}

3. Sorting Complexity

some items:

computational model: allowed operations, (ex. decision tree for compare based sorting).
cost model: operation counts.
upper bound: cost guarantee (ex. NlgN for mergesort). .
lower bound: limit on cost guarantee for all algorithms (no algorithm can do better).
Optimal algorithm: algo with best cost guarantee. (upper bound=lower bound)

证明lower bound 的方法很有意思:

lower bound for sorting

(binary) decision tree for the case of 3 distinct items:

each possible ordering is a leaf of the decision tree → there are N! possible orderings → tree height should be lg(N!) = NlgN (cf. https://en.wikipedia.org/wiki/Stirling%27s_approximation)
proposition. **
Any compare-based sorting algorithm must have at least lg(N!)~NlgN compares in the worst case. (for N distinct keys)
proof.

binary tree of height h has at most 2^h leaves
N! possible orderings → at least N! leaves

So, lower bound for sorting = ~NlgN ⇒ mergesort is an asymptotical optimal algorithm. (In terms of time complexity, the shortcoming of mergesort is always the extra space usage)

但是并不是说NlgN是所有排序问题的下界:

4. Comparators

Java mechenism for comparing same data on different ways.

Comparable interface

→Sorting using natural order for a data type.
public class Data implements Comparable{
public int compareTo(Date that){
//...}
}

Comparator interface

→Sorting using an alternative order. (total order property is required...)

public interface Comparator<Key>{   
    int compare(Key v, Key w);   
}

Comparator can be passed as arguments in java system sort:

**Using Comparators in our sorting algos **
Use another signature:

change Comparable to Object
add Comparator in arguments

public static void sort(Object[] a, Comparator comparator);
public static boolean less(Comparator c, Object v, Object w);

public static void exch(Object[] a, int i, int j);

**Implementing a comparator **
Add static comparator to a class:

In the declaration of a class, define an inner class that implements the Comparator interface,
Then declare an instance of this inner class as a static final variable...
note: the inner class should be static also.

以上的方法目的是为某个类提供预先定义好的comparator(比如String.CASE_INSENSITIVE_ORDER), 另外也可以直接再声明一个类作为comparator, 然后实例化这个类传进去, 就像之前做过的那样.

*Application: Graham scan algo for convex hull **

We can get the result of compare by calling ccw(a,b,c) :
ccw(p,q1,q2)=true ⇔ q2>q1 in terms of polar angle wrt p. (别忘了p是y坐标最小的点, 否则还要讨论角坐标为负数的情况, 见下图). *

5. Stability

Typical application: first sort by student name then by section.

def. :stable"
A stable sort preserves the relative order for items with equal keys.

Insertion sort and mergesort are stable, whereas selection sort and shell sort are not. (And always need to carefully check code: "less than" vs. "less than or equal to").

insertion sort:
stable
proof: we never move equal items pass each other: if(less(a[j], a[j-1])) exch(...)

**selection sort: **
not stable
counter example:
when exchanging A1 and B1, we move B1 behind B2

**shell sort: **
counter example: long-dist exchanges

**merge sort: **
stable
proof: suffices to verify that merge operation is stable, if keys are equal, always take element from left subarray.

[Algorithms I] Week 2-2 Elementary Sorts

2015-08-18T00:00:00+02:00

1. Introduction

rearanging array of size N into ascending order
test client code: Insertion.sort(a);

sort any datatype

callback

callback = reference to executable code
i.e. passing functions as argument to sort() method
sort() function calls object's compareTo() method

→ implement the Comparable interface:

    public class XX implements Comparable<XX>{
    ...
    }

the interface:

    public interface Comparable<Item>{
        public int compareTo(Item that);
    }

compareTo():
return -1 (if this<that)/+1/0; needs a total order. → in the sort() implementation:
has not dependencies on type of data.

    public static void sort(Comparable[] a){
        if(a[i].compareTo(a[j])>0)...
    }

helper functions

less

private static boolean less(Comparable v, Comparable u){ returnv.compareTo(u)<0; }
exch

private void exch(Comparable[] a, int i, int j) { Comparable swap = a[i]; a[i] = a[j]; a[j] = swap; }
isSorted

test if sorted if algo passes the test using only less ant swap, then it's correct.

2. Selection Sort

Idea: each time: find the minimum from the remaining items. a[min] is the smallest element to right of a[i] ⇒ swap a[i] and a[min] (elements to left of i are sorted)

invariants

entries to the left of i are in sorted order, and are fixed (in final position) ever since
no entry to the right of i is smaller than any entry to the left of i

implementation

public class SelectionSort extends AbstractSorting{
//...
public static void sort(Comparable[] a){
    for(int i = 0; i<a.length; i++){
        int min = i;
        for(int j = i+1; j<a.length; j++)
            if(less(a[j],a[min])) 
                min = j;
        exch(a,min,i);
    }
}
}

analysis

proposition: selection sort uses N-1 + N-2 + ... + 1 = ~N^2/2 compares, and N exchanges. → quadratic time

insensitive to input: quadratic time even if input is already sorted.
data movement is minimum: linear time of exchanges (every exchange puts an item to its final position)

3. Insertion sort

quite different performance characteritics than selection sort.

Idea: In iteration i: move all entries larger than a[i] to its left.

invariants

entries to the left of i are in ascending order (but not in final position)
entries to the right of i are not yet been seen

implementation

publc class InsertionSorting extends AbstractSorting{
    public static void sort(Comparable[] a){
        for(int i=1; i<a.length; i++)
            for(int j=i; j>0; j--){
                if(less(a[j],a[j-1]))
                    exch(a,j,j-1);
                else break;
            }
    }
    //...
}

analysis

proposition (average case):
(the performance on average — for randomly sorted array )

proof: expect each entry to move halfway back

best case and worst case

best case if array already sorted min ascending order: N-1 compares, 0 exchanges.

worst case if array sorted in descending order: every element goes all the way back → 1/2N^2 compares, 1/2N^2 exchanges

partially sorted arrays

def." inversion" an inversion is a pair of entries that are out of order.

def. "partially sorted"
An array is called partially sorted if the number of inversions is <= cN. *

proposition.
Insertion sort runs in linear time for partially sorted array. proof. number of exchanges = number of inversions. * number of compares = number of exchanges + N-1*

4. Shell Sort

First non-trival sorting methode: an improvement of insertion sort.

def. "h-sorted array"
an array is h-sorted if every h-interleaved subarray is sorted. (h=1: just a sorted array)

Idea: move entries >1 position at a time by h-sorting the array, then decrease h.

use decreasing sequences of value h:

implementation

How to h -sort
simply insertion sort with stride length=h.
why insertion sort:
for big h: small subarray
for small h: nearly in order

proposition
A g-sorted array remains g-sorted after h-sorting it.

(subtle to prove...)

which sequence of h to use 3x+1 sequence proposed by Knuth. 1,4,13,40....

public class ShellSort extends AbstractSort{
    public static void sort(Comparable[] a){
        int h = 1, N=a.length;
        while(h<N/3) h = h*3+1;//find the beginning h (N>h>N/3)
        while(h>=1){//performs h-sort
            for(int i= h;i<N;i+=h)
                for(int j = i;j-h>=0;j-=h)
                    if( less(a[j],a[j-h]) )
                        exch(a,j,j-h)
                    else break;
            h = h/3;
        }
    }
    //...
    private static boolean isHsorted(Comparable[] a, int h) {
        for (int i = h; i < a.length; i++)
            if (less(a[i], a[i-h])) return false;
        return true;
    }
}

每次hsort, 外围的循环是for(int i= h;i<N;i+=h), 需要理解一下: i移动一次以后, 进行的是另一个subarray 的插入排序, 当移动到N-1的时候所有subarray的插入排序才结束. (也就是说不是先完成一个subarray的插入排序再完成另一个, 这些是插入排序是同步进行的)

analysis

proposition (for worst case )

→ better than quadratic time !

property (found in practice)

of compares < Cte * N * (# of h used )

→ #compares < NlgN * Cte

*accurate model has not been discovered *

(所以shellsort在实际使用中几乎和快速排序一样快! — 尽管没有数学证明来保证)

why we are interested in shell sort useful in practice:

fast for medium sized arrays (beat even the classical sophistiated algorithms)
tiny code volumn (used in embeded systems)

lead to interesting questions for 50 years:

asymptotic growth rate ?
best sequence of h ?
average case performance ?

5. shuffling

shuffle array using sort

one way to shuffle an array:

for each array entry, generate a random real number
sort the array of real numbers
⇒ the original array is shuffled !

proposition
this shuffle sort produces a uniformly random permutation of input array
drawback: cost for sorting...

Goal: get uniformly random permutation in linear time.

Knuth shuffle

algo:

for i = [0,N):
- r = rand( [0~i] ) or rand( [i, N-1] )
- swap a[r] and a[i]

implementation:

public static void shuffle(Object[] a){
    for(int i=0;i<a.length;i++){
        int r = StdRandom.uniform(i+1);
        exch(a,r,i);
    }   
}

proposition
Knuth algo produces an uniformly random permutation of input array.

proof.
Sufficient to prove that, for card i and position j, the proba(card i comes to position j) = 1/N.

if i<=j, P = 1/j * j/(j+1) * (j+1)/(j+2) * ... * (N-1)/N
if j<i, P = 1/i * i/(i+1) * (i+1)/(i+2) * ... * (N-1)/N

CQFD.

example: online poker

https://www.cigital.com/papers/download/developer_gambling.php ←那个扑克网站已经被黑出翔了...

bugs:

r never get 52 (52th card never moved)
r = rand(N) instead of rand(0~i), → shuffle not uniform
random() uses 32bit seed: only 2^32 possible shuffles, 2^32<52!
seed = millisec from midnight, ~86*10^6 suffles

6. Convex Hull

application of sorting for the field of computational geometry.

convex hull

smallest polygoneenclosing all N points.

input: N points
output: sequence of vertices in counterclockwise (ccw) order.

application: robot motion planning; farest pair.

geometric properties:

can traverse convex hull by making only ccw turns
let p be the point with lowest y-coord, wrt p, vertices appear in increasing order of polar angle.

Algo

Graham scan algorithm:
* choose p with smallest y coord
* sort points by polar angle with p
* consider points in order (stack is used), discard unless creates a ccw turn.

CCW
given three points a b c, returns if a→b→c is a CCW turn.

(assumption: no 3 points on a line)
⇒ calculate cross product of ab and bc ⇒ determinants!

area>0 ⇔ CCW

implementation

public class Point2D{
private double x,y;
public static boolean ccw(Point2D a,Point2D b,Point2D c){
    double area2 = (b.x-a.x)*(c.y-a.y) - (b.y-a.y)*(c.x-a.x);
    return area2>0;
}

}

convex hull:

public static Stack<Point2D> GrahamScan(Point2D[] p){
    //* assumes that points are sorted by polar angle in p[]
    Stack<Point2D> hull = new Stack<Point2D>();
    hull.push(p[0]);
    hull.push(p[1]);
    for(int i=2;i<p.length;i++){
        Point2D b = hull.pop(), a = hull.peek(), c = p[i];
        while(!Point2D.ccw(a,b,c)){
            b = hull.pop();
            a = hull.peek();
        }
        //now a,b,c makes a ccw turn:
        hull.push(b);
        hull.push(c);
    }
}

running time: NlgN for sorting and linear for the rest.

[Algorithms I] Week 2-1 Stacks and Queues

2015-07-09T00:00:00+02:00

fundamental data types: stacks and queues operations: insert, remove, test empy, iterate,...

module programming: seperate interface and implementation

1. Stacks

ex. a stack of strings

API:

public interface StackoOfStrings{ void push(String item); String pop(); boolean isEmpty(); //int size(); }

implementation 1: using a linkedlist

insert/remove from the top of the linkedlist

inner class

class ListNode{
    String item;
    ListNode next;
}

implementation

public class LinkedStackOfStrings implements StackoOfStrings{
    class Node{
        String item;
        Node next;
        Node(String item, ListNode nxt){...}
    }
    private Node first;

    public LinkedStackOfStrings(){
        first = null;
    }
    public void push(String item){
        Node nd = new Node(item,first);
        first = nd;
    }
    public String pop(){
        String firstItem = first.item;
        first = first.next;
        return firstItem;
    }
    public boolean isEmpty(){
        return first==null;
    }
}

complexity: const time for every operation

array implementation

use array (of length N) to store items → defect: stack has limited capacity
keep a pointer top: pointing to the next empty space to push (top 的定义很重要)
problems of the array implementation:
1. underflow: pop from an empty stack
2. overflow: size larger than capacity ⇒ resizing
3. loitering: holding a ref to an obj which is no longer needed: ex. return s[top--] java system will not know that s[top] is no longer needed ⇒ have to clear it explicitely ⇒
  
  String item = s[top--]; s[top]=null; return item

implementation (containg resizing array operations)

public class ArrayStackOfStrings implements StackOfStrings{
        private String[] s;
        private int top=0;
        public ArrayStackOfStrings(){// to be tuned
            s = new String[1];//initial capacity=1
        }
        public boolean isEmpty(){
            return top==0;
        }
        private vois resize(int capacity){//helper functoin
            String[] s2 = new String[capacity];
            for(int i=0;i<top;i++)
                s2 = s[i];
            s = s2;
        }
        public void push(String item){
            if(top==s.length)//doubling size 
                resize(s.lenth*2);
            s[top++]=item;
        }
        public String pop(){
            String item = s[--top];//NOT top--!
            s[top]=null;
            if(top>0 && top==s.length/4) //top>0 is necessary
                resize(s.length/2);
            return item;
        }
}

2. Resizing Arrays

resolving the overflow pb: grow and shrink the array → need to copy all items when changing array size ⇒ pb: ensure that sizing happens infrequently

resizing strategy

repeated doubling:

(initial capacity=1) when array is full, double the size

amortized complexity for inserting N:

N+(2+4+8+...+N) ~3N

shrinking array

⇒ shrink the array by half when array is 1/4 full not half full → thrashing will happen if push-pop-push-pop when array is full

[invariant]: array always 20%~100% full
complexity:

in an amortized sense, will be constant

proposition: from empty stack, M operations of push/pop taked time propotional to M

comparison: resizable array vs linkedlist

linkedlist implementation:

operations takes const time even in worst time extra time and space for dealing with linkes

resizing array implementation:

operation taked const amortized time but in worst case takes linear time (ex. to be evited for critical systems) less wasted space

3. Queues

FIFO data structure API

public interface QueueOfStrings{
    void enqueue(String item);
    String dequeue();
    boolean isEmpty();
    //int size();
}

linked list implementation

maintain first and last node pointers: pointing to 2 points of queue (first for dequeue, last for enqueue )

→ take care of corner cases: - empty queue: first is null (and last is also null) - just one item in queue: first and last point to the same node (总之first和last的定义很重要)

    public class LinkedQueueOfStrings implements QueueOfStrings{
        class Node{...  }
        private Node first,last;
        public LinkedQueueOfStrings(){
            first = null;
        }
        public void enqueue(String item){//same as push
            Node nd = new Node(item,null);
            if(isEmpty()){
                last = nd;
            first = last;
            }
            else{
                last.next = nd;
                last = nd;
            }
        }
        public String dequeue(){//same as pop in stack
            String firstItem = first.item;
            first = first.next;
            if(isEmpty())
                last=null;
            return firstItem;
        }
        public boolean isEmpty(){
            return first==null;
        }
    }

resizing array implementation

maintain head and tail: head is the queue head, tail is the next empty position for the next element to enqueue → trick: head and tail should take mod capacity + resizing array

不知道写的对不对:

    public class ArrayQueueOfStrings implements QueueOfStrings{
        private String[] q;
        private head=0,tail=0;
        public LinkedQueueOfStrings(){
            q = new String[1];//init capacity
        }
        public boolean isEmpty(){
            return head==tail;
        }
        private void resize(newsz){
            q2 = new String[newsz];
            int i = head,j=0;
            while(i!=tail){
                q2[j++] = q[i];
                i=(i+1)%q.length;
            }
            q = q2;
            head=0;
            tail=j;
        }
        public void enqueue(String item){
            if( (tail+1)%q.length==head )
                resize(q.length*2);
            q[tail] = item;
            tail = (tail+1)%q.length;
        }
        public String dequeue(){
            String firstItem = q[head];
            head = (head+1)%q.length;
            int sz = (tail-head)%q.length;
            if(sz>0 && sz==q.length/4)
                resize(s.length/2);
            return firstItem;
        }
    }

4. Generics

queues/stacks for other types of data ⇒ generics 泛型(java 1.5 才引进泛型机制...) use type paramater→ avoid casting, and discover type mismatch errors at compile time

    public interface Stack<Item>{
        public void push(Item item);
        public Item pop();
        public boolean isEmpty();
    }

a pb with array implementation

java不支持创立泛型数组 generic array creation is not allowed. 不可以new 一个泛型数组! s = new Item[capacity];会报错
⇒ use an ugly cast: s = (Item[]) new Object[capacity]; (will get warning: "unchecked cast" → java被黑了... )

autoboxing for primitive types

each primitive type has a wrapper class ex. int ↔ Integer autoboxing: automatic cast between a primitive type and its wrapper class.

(syntactic sugar 语法糖 i.e. 对语言功能没有影响只是方便使用)
btw: https://zh.wikipedia.org/wiki/%E8%AF%AD%E6%B3%95%E7%B3%96 (居然还有语法盐和语法糖精......)

5. Iterators

Interface

support iteration over stacks and queues, without revealing the internal representation of stack/queue ⇒ implement the Iterable interface

Iterable interface: can return an *Iterator *

public interface Iterable{
    Iterator<Item> iterator();
}

Iterator interface: hasNext() and next() interface

public interface Iterator<Item>{
    boolean hasNext();
    Item next();
    void remove();//optional, bad practice to use it
}

to make a data structure Interable → elegant client code

how-to:

implement Iterable interface
write a private inner class XXIterator that implment the Iterator interface.

ex.

Bag data structure

Supports adding and iterating through without caring about the order.
API:

    public class<Item> Bag implements Iterable<Item>{
        public void add(Item);
        int size(); 
    }

can be implemented by stack or queue(without pop/dequeue)

6. Applications

Java collections library

List interface: java.util.List

implementations: ArrayList, LinkedList.

pb with the java's implementation of stacks and queues:

Stack class also implements List interface (get(), remove(), contains() are implemented);
Queue is an interface rather than a class...
⇒ **poorly designed API **

Stacks applications

function calls:

recursion: can always use an explicit stack to remove recursion

arithemic evaluation (Dijkstra)

四种类型: 左括号, 右括号, 数字, 算子

最后一行应该是value stack.

⇒ 后缀表达式, 逆波兰式......

[Algorithms I] Week1-Lab: Percolation

2015-07-07T00:00:00+02:00

model & problem

(原文描述太啰嗦了)
A system using an N-by-N grid of sites.
→ Each site is either open or blocked.
→ A full site is an open site that can be connected to an open site in the top row via a chain of neighboring open sites. (这个full的定义有玄机而且导致后面写程序时有个问题, 看论坛想了半天才想出来, 见后文.)
→ We say the system percolates if there is a path of connected open sites form the top row to the bottom row.

⇒ pb: if sites are independently set to be open with probability p, what is the probability that the system percolates?

→ When N is sufficiently large, there is a threshold value p such that when p < p a random N-by-N grid almost never percolates, and when p > p, a random N-by-N grid almost always percolates.
→ No mathematical solution for determining the percolation threshold p has yet been derived.
⇒ Your task is to write a computer program to estimate p*.

Method

API:

public class Percolation {
   public Percolation(int N)               // create N-by-N grid, with all sites blocked
   public void open(int i, int j)          // open site (row i, column j) if it is not open already
   public boolean isOpen(int i, int j)     // is site (row i, column j) open?
   public boolean isFull(int i, int j)     // is site (row i, column j) full?
   public boolean percolates()             // does the system percolate?
   public static void main(String[] args   // test client (optional)
}

Corner cases: the row and column indices i and j are integers between 1 and N. 1≤i,j≤N

if i/j out of range: java.lang.IndexOutOfBoundsException if N<=0 in constructor: java.lang.IllegalArgumentException

Performance requirements: N2 for constructor, const for other operations
Monte Carlo simulation
- all sites init to be closed
→ randomly choose a blocked site (i,j) and open it → repeat until percolates ⇒ the fraction of opened sites is an estimation of p*
ex. 20*20 grid, when percolated:

⇒ estimated p* = 204/400=0.51

repeat the estimation for T times, get T estimations

→ get mean and std:

→ 95% 置信区间:

create API for this simulation:

public class PercolationStats {
   public PercolationStats(int N, int T)     // perform T independent experiments on an N-by-N grid
   public double mean()                      // sample mean of percolation threshold
   public double stddev()                    // sample standard deviation of percolation threshold
   public double confidenceLo()              // low  endpoint of 95% confidence interval
   public double confidenceHi()              // high endpoint of 95% confidence interval
   public static void main(String[] args)    // test client (described below)
}

-if N ≤ 0 or T ≤ 0: java.lang.IllegalArgumentException
-main() : takes two command-line arguments N and T
⇒ performs T independent computational experiments on an N-by-N grid, and prints out the mean, standard deviation, and the 95% confidence interval for p*.
(Use standard random from our standard libraries to generate random numbers; use standard statistics to compute the sample mean and standard deviation.
Here is the algo API: http://algs4.cs.princeton.edu/code/index.php)

Code

注意一定要用它们提供的那些库, 否则自己写的话代码就长了....
shuffle, mean, stddev什么的直接用他们的函数库就可以做到.
http://algs4.cs.princeton.edu/code/index.php
另外UF也是用他们写好的, WeightedQuickUnionUF.

按照提示, 除了格子的N^2个节点以外再增加两个节点: 顶部和底部的虚拟节点. 这里写的时候注意一开始也是不恩能够把它们与第一行/最后一行相连的 — 要在一个格子open以后再相连.

backwash问题

这次题目有一点比较困难就是, 需要实现isFull()函数, 这个函数判断一个格子(i,j)是否和顶部相连. 这里如果直接用UF的connected()判断是否和顶部虚拟节点相连的话是有问题的, 如下图:

白色格子表示格子是open的, 蓝色格子表示格子是open并且是full的(i.e. 和顶部相连的), 左边图片里的状态是对的, 右边图片里底下部分的格子状态则不对: 如左下角的格子, 其实是没有和顶部联通的, 如果我们用两个虚拟节点的话, 由于底部虚拟节点和顶部虚拟节点相连, 所以和底部虚拟节点相连的左下角部分就被判断成了full的.

这个问题一开始我以为可以很简单解决, 后来发现没那么容易... (注意题目还要求isFull()也要在常数时间给出结果).
一个不优雅的办法是, 建立两个UF, 一个用来判断percolation, 另一个UF里没有底部虚拟节点所以可以专门用来判断isFull().

这样解决的话使可以通过测试, 不过非常不好看, 另外一个UF的内存占用是8N^2(内部有size[]和id[]两个int数组), 比较大.

在论坛上找了半天, 看了一些人的分享终于想到了这个非常妙的办法:

UF只建立顶部虚拟节点, 不建立底部虚拟节点.
判断isFull只需要用UF的connected()一下就好了
问题是怎么判断percolation:
a. 建立一个数组 boolean connectedToBottom[], 指示某一点是否和底部相连
b. trick在这里: 不必修改一个联通分支的所有点的connectedToBottom的值, 只需要修改联通分支的root(UF的find)即可. 在进行union的时候先查看两个component的root是不是连到底部, 然后有一个连到底部的话, 在union以后把合并后的联通分支的connectedToBottom状态改为true即可
c. 然后判断percolate: 先找到顶部虚拟节点锁在component的root, 然后看这个root是否连到底部即可!

这样用一个boolean数组(N^2内存)代替了一个新的UF(8N^2内存), 而且实现也更加优雅.
非常有意思的练习...

[Algorithms I] Week 1-2 Analysis of Algorithms

2015-07-02T00:00:00+02:00

1. Introduction

2. Observations

ex. 3-SUM pb
given N distinct numbers, how many triples sum up to 0? (pb related to computatioal geogtry)

brute force method:

for(int i=0;i<N;i++)
    for(int j=i+1;j<N;j++)
        for(int k=j+1;k<N;k++)
            {if(a[i]+a[j]+a[k]==0)
                count++;
            }

mesuring running time:

stdlib.jar里面提供了一个Stopwatch类用于记录运行时间.

log-log plot

T(N) = running time for input of size N
log(N)-log(T(N)) plot:
often get a straight line — power law

doubling ratio:

(for checking the power law relationship, checking the power order)
each time double the size of input, then take log of the time ratio of 2 runs: log( T(2N)/T(N) )

3.Mathematical Models

total running time: sum of cost*frequency of operations
cost of some basic operations:
- array allocation: c*N (because all array entries have to be set to 0/false/null)
- string concatenation: c*N (proportional to the length of string !)
simplification

crude analysis
ignore lower terms tilde notation

estimating discrete sum by relaxation

Replace the sum with an integral, and use calculus — 很机智...

4. Order of Growth Classification

(discard the leading coefficient when considering the growth order)

only a small set of growth functions:

1, logN, N, NlogN, N^2, N^3, 2^N

exemples:

binary search ⇒ logN
divide and conquer ⇒ NlogN
exhaustive search ⇒ 2^N

practical performance:

ex. binary search

public int binearch(int arr[], int key){//arr[] already sorted int lo=0,hi=arr.length; while(i<j){ int m = (lo+hi)/2; if(arr[m]==key) return m; else if(arr[m]<key) lo=m+1; else hi=m-1; } return -1; }

(→ Bug in Java's Arrays.binarySearch() discovered in 2006......)
→ invariant: if key in arr, arr[lo]<=key<=arr[hi]
proposition. binary search uses at most logN+1 compares to search a sorted array of size N. pf. ** denote T(N) := nb of compares for array with size <=N
→ T(1)=1
→ recurrence relation: T(N)<=T(N/2)+1
⇒ T(N)=logN

a faster 3-SUM

→ first sort the array (~NlogN)
→ for any pair a[i] and a[j], do binary search for -(a[i]+a[j]) ~(N2LogN)
⇒ reduce from N3 to N2logN ! (for 8k numbers, running time goes from 51s to 0.96s)

5. Theory of Algorithms

types of analysis

-best case -worst case -average case(random input, "expected cost")
notations

big Theta/big O/big Omega

- big O: upper bound → * once a specific algo is found, find an upper bound
- big Omega: lower bound → proove that no algo can do better
- big Theta: symptotic growth (same order, optimal algo) → lower and upper bound match*

⇒ in this course: use tilde notation: contain leading constants for highest order term

6. Memory

KB: 2^10 bytes
MB: 2^20 bytes (1 million) GB: 2^30 bytes (1 billion) 64-bit machines: 8 byte pointers

typical memory usage:

for primary types:

for arrays (with array overhead=24bytes) :

Obj overhead: 16 bytes (obj的大小=16+obj内部filed的大小)
references: 8 bytes (ex. inner class has a ref to encolsing class)
padding: each obj uses a multiply of 8 bytes (obj大小=8 bytes的整数倍)

[Algorithms I] Week 1-1 Union-Find

2015-06-27T00:00:00+02:00

1. Dynamic Connectivity pb

pb statement

a set of N obj, indexed by 0,1,...,N-1 ⇒ * UNION: connect objects void union(int p, int q) * FIND: is there a path connecting 2 obj? boolean connected(int p, int q)

ex:

connect components(联通分支): max set of obj that are mutually connected.

UF API

union(p,q): connect 2 obj
connected(p,q): test if p and q are connected
find(p): find the component id of p
count(): nb of components

注意: 命名不是很好, 这里的find()函数不对应FIND query, connected()函数才是真正的FIND query, find()函数是为了connected()函数而做的一个辅助函数(find(p): return the root of the node p ) 在connected里就可以调用find: return find(p)==find(q)

应该是interface更好一些...

public interface UF{
void union(int p, int q);
boolean connected(int p, int q);
//int find(int p);
//int count();
}

测试client:

2. Quick Find

"eager approach"

data structure

⇒ an int array id[] initialized to id[p]=p for all p interpretation: id[p] = component id of obj p ⇒ p and q are connected iff* id[p]==id[q] (ie. find very fast) 没有用find()函数

UF operations

FIND:

connected(p,q): very fast, just check id[p] and id[q]

UNION:

when merging 2 components : union(p,q): id[p]=id[q](总是让第一个参数p的id变为第二个参数q的id),
⇒ then have to modify all entries with id equal to id[p] !
⇒ too many entries to change

implementation

(class QuickFindUF implements UF)

complexity:

FIND: cte
UNION: lin...

if N obj + N unions ⇒ quad time !
btw, 程序运行速度: ~10^9/s

3. Quick Union

"lazy approach"

data structure

⇒ also an int array id[] considering a set of trees, 此时每个联通分支都是一个tree
interpretation: id[p] = parent index of obj p (觉得这个数组叫做father更好....)
⇒ p is a root node iff id[p]==p

UF operations

FIND:

connected(p,q):check if root of p == root of q

UNION:

union(p,q):just set p's root to be child of q's root (把第一个参数p的那棵树放入第二个参数q的树的根节点作为子树)

root():

前两个的操作都需要一个函数查找一个节点的root, 需要写一个函数实现, 也很简单, 一路找parent即可:

private int root(int p){
    while (p!=id[p]) p=id[p];
    return p;
}

implementation

complexity

in the worst case (all elements is in a list form), root() is ~N, so:

FIND: lin
UNION: lin

quick find和quick union的问题:

4. Quick Union Improvements

improvement1: weighting

keep track of tree size ⇒ balance by taking the small tree be a child of the large tree

⇒ add an extra array: sz[] sz[i] is the size of the tree with root i

implementation

(数组int sz[] 初始全部为1)

依然需要root()函数.

private int root(int p){ while(p!=id[p]) p=id[p]; return p; }
FIND

public boolean connected(int p, int q){ return root(p)==root(q); }
UNION

public void union(int p, int q){ int rp = root(p), rq=root(q); if(rp==rq) return; // if (sz[rp]<sz[rq]){ id[rp]=rq; sz[rq]+=sz[rp]; } else{...} }

complexity

FIND: proportional to depth of p and q in their tree UNION: const if p and q are root

proposition

the max depth of weightedQuickUnion is lgN

[pf] considering a node x, in tree T1, dep(x) is x's depth in its tree.

→ dep(x) will increase by 1, iff T1 is merged into another tree T2 (and by the algo, shoud have |T1|<=|T2| )
→ x's tree's size become |T1|+|T2| >= 2|T1| ⇒ everytime dep(x) increased by 1, x's tree's size will at least double*
at first dep(x)=1, if dep(x) increases lgN times, the size of the tree will be >= N
CQFD.

so the root() function takes only lgN time. conclusion: both UNION and FIND will be in lgN time.

improvement 2: path compression

imporve the root() function:

when looking for root of a node ⇒ link all nodes in the path up to the root.
⇒ just a constant extra time compared to old implementation.

2 pass implementation:

private int root(int p){ int r = p; while(r!=id[r]) r=id[r]; while(p!=r){ int t = p; p=id[p]; id[t]=r; } return r; }

flatens the tree greatly.

single pass implementation:

just make all other node point to its grandparent (halving the path length) ⇒ not as flatening as before, but in practice will almost be the same. just one extra line of code: private int root(int p){ while(p!=id[p]){ id[p] = id[ id[p] ]; p=id[p];
} return p; }

complexity

(for weighet quick union with path compression — WQUPC)
very very small:

lg*() function: "iterated log function", lg(N) = the number of time to take log to get to 1 lg()几乎可以看成常数了:

ex. lg(65536) = 4 (x^16=65536)
because: lg(65536)=16 ; lg(16) = 4; lg(4)=2; lg(2)=1.
⇒ N obj, M unions will take (almost) linear time*

(有人证明了不存在理论上linear的算法. )

conclusion: both UNION and FIND will be in constant time.

summery

上面这个表格好像quick union的部分有问题? 最坏情况下应该是N+MN吧??
书上是这么写的:

WQUCF reduce 30 years to 6 seconds.

5. Union Find Application

percolation
dynamic connectivity
Kruskal MST algo
Games (GO)
.......

percolation

model:

NN grid of sites
⇒ each site is open with proba=p
⇒ sys percolate* iff bottom and top are connected by open sites.

question: the percolation probability as a function of p (phase transition)

nobody knows how to get the threshold mathematically
⇒ run simulations to find out the phase transition threshold.

Monte Carlo simulation

→ all sites initilized to be closed
→ randomly open sites one by one
→ when the sys percolates, the vacancy percentage is an estimate of p
*(run above simulation for millions of times)

implementation

N^2 sites, named 0 to N^2-1

add 2 more vertual sites: one on top, one on bottom

openning a site: union to adjcent open sites (at most 4 unions)

运筹的力量: 用线性规划解决Google 2014 HashCode问题

2015-02-02T00:00:00+01:00

INF580(programmation par contraintes) 大概是在X学到的最有用的一门课, 它让我能够用把运筹学(MAP557)里学到的东西和计算机结合起来: 用电脑的力量解决(大规模)运筹问题.

这门课的projet是去年巴黎谷歌举行的一个比赛的题目: 最优化谷歌街景拍照小车的路线. 做这个projet的三周里, 我和Manu从一开始信心满满, 到中间一筹莫展, 再到后来柳暗花明, 以及最后乘胜追击终于在今晚得到了近乎完美的解答, 非常精彩, 这里特意一记.

问题描述

谷歌那次比赛的题目在这里(我们做的是Main Round的题目): https://sites.google.com/site/hashcode2014/tasks

简单来说, 就是已知巴黎的道路信息, 设法用八辆车(每辆车的行驶时间有限)从巴黎谷歌出发, 尽可能多的走遍巴黎的所有街道, 参赛者给出这些车的路线, 他们的分数就是这八辆车走过的街道的长度之和(重复走的街道不算分).

去年四月份我们也参加了这个比赛, 不过当时纠结于如何设计每辆车的路线, 最后只是用了贪心算法, 再加上一点点的随机, 得到的结果并不好... 当时ENS的人包揽了前三名, 而且比赛后进一步把分数刷到了满分: 他们的路线可以把所有街道都跑遍.

这学期学了INF580以后, 手里有了JaCoP以及AMPL等"重型武器", 我们经过一番努力也得到了满分, 而且用了几乎最少的时间. 代码放在了: https://github.com/X-Wei/INF580_HashCode2014 (里面还有个pdf的报告, 比这篇博客里说的精炼).

I. 初步建模

线性规划(LP)模型

站在一个更高的角度上考虑这个问题, 从一个线性规划的角度看, 给定一个图G(V,E), 目标是最大化走过的街道长度, 如果用对每条街道e, 定义一个x_e,: 当一条街被经过的时候使它为1, 否则为0, 那么目标函数就是一个x的线性组合(系数就是对应街道的长度).

但是如果一条街被经过了大于一次, x_e也只能是1, 所以再引入一个变量y_e: 用来表示一条街被经过的次数. 所以x_e可以看成是y_e的indicatrice函数: , 这一点可以通过增加线性约束: x_e<=y_e 即可实现, 因为x_e的取值范围是{0,1}.

不过一条街可能从两个方向被经过(G里面的边是有向的), 如果e'代表反方向的e, 那么还应该增加约束: x_e+x_e'<=1 就好了.

接下来y要满足的就是所有的y要组成一个路径(path), 对于既不是起点又不是终点的一个节点v, 还是很好写的, 类似flot: sum(进入v的y_e) = sum(从v流出的y_e).

对于起点v_start来说, 其实也不难: sum(进入v_start的y_e) = sum(从v-start流出的y_e)+1.

但是对于终点的话, 有个问题是我们不知道车会停在哪里! Manu太聪明了, 对每一个节点v引入了另一个变量f_v,取值范围也是{0,1}, 如果在节点v停下来, 那么f_v等于1, 否则就是0. 所以上面的约束可以写成: sum(进入v的y) = sum(从v流出的y_e)+f_v. 另外别忘了只能停在一个节点, 所以再加约束: sum(所有的f_v)=1.

以上的目标函数以及约束, 虽然数目庞大, 但是都是线性的, 所以是一个整数线性规划问题, 写成数学形式就是:

从solution得到路径

一旦上面那个线性规划问题解决了, 我们得到的将会是那些变量x,y,以及f, 但是需要从中提取一条路径最后才能作为solution!!

这个问题实际上可以归结为: 给定一个一笔画的曲线(其实就是那些y: 可以想象把每条边都复制y次, 得到的就是这么一个曲线了), 找出一个可以将其一笔画出来的路径(这条路径也叫"欧拉路径", 没"汉密尔顿路径"有名). 聪明的Manu很快想出来一个算法(我们叫它"Orsini算法"): 大意是当走到底没有走完的时候, 把那些错过的路径夹在原路径的中间就好了.

算法描述如下:

i) 从起点开始出发走, 把走过的边涂上颜色(以后不能再走), 只要还可以继续走(还有没有着色的边可走)就一直走下去, 一直到无路可走. (应该会停在f=1的地方), 得到路径p0.

ii) 如果所有边已经被走过了: over. 如果没有: 找一个没有走过的边与p0的交叉点: v(如果找不到的话说明有问题: 后面就遇到这个问题了), 然后执行(iii)

iii) 从v出发, 在进行类似(i)的操作得到一条路径(应该是一个环), 然后把p0从v那里劈开, 把这个环塞到中间组成新的p0, 然后再执行(ii).

这样就可以得到对应的路径了.

一开始的想法

以上的建模是针对一辆车的, 我们一开始的想法是: 对一辆车进行这个操作, 得到结果以后更新一下G(把那些已经走过的街道的长度设为0, 然后再走下一辆车... 这样跑8次就能得到最优解了.

一切看起来似乎都非常完美......

II.遇到的问题

计算能力问题

写好了AMPL的程序以后, 我们遇到的第一个问题是: 这么大的问题, 一般电脑算不出来(曾经用glpk让电脑跑了一夜, 还是没有得到结果).

后来问老师, 老师说glpk并不是非常高效的求解器, 然后推荐给了我们一个非常给力的网站: NEOS ! 这个网站可以让人上传AMPL程序, 然后用它们的服务器跑, 而且使用的求解器也是商用的, 比开源求解器快的不是一个数量级(我们发现最快的求解器是Gurobi, 十分给力).

这样一来计算的问题就解决了, 但是我们很快发现一个更严重的问题, 是我们的建模里的一个严重缺陷...

subtour问题

我们兴冲冲的拿NEOS的结果跑Orsini算法的时候, 发现总是报错: 也就是在第二步, 程序无法找到一个和p0有交叉点的边: 也就意味着我们的模型得到的路径并不是只有一个connected component!!! 也就是说, 我们得到的解其实是一条从起点到某个终点的路径, 外加很多和这个路径没有交点的圈圈(subtour)!

是啊, 这个问题很类似旅行商问题(TSP), 而TSP的困难之一就是要解决subtour的话需要加入2^n个新的约束.... 2^10000个约束? impossible...

III. subtour问题的"解决"

各种纠结

在一次PC上有一道题目介绍了TSP subtour问题的一种建模方式, "potentiel"建模, 可以防止加入2^n个约束: 为每个节点v引入新的变量u_v,u_v代表了节点v被访问的顺序, 约束做的非常巧妙, 是这样的:

这样, 当x_ij是1的时候, 就保证u_j比u_i大1, 而当x_ij是0的时候, 这个约束则非常松弛, 几乎相当于没有.

不过这个方法套到我们这个projet的话也有问题: 那就是每个节点只能有一个u_v, 所以每个节点只能最多访问一次.... 不过我们很快想到了办法: 把所有节点复制一个(复制"一层"), 然后每条边的话也进行复制, 同时加上那些连接各层的边.

举个例子, 原来的一条边是: (u, v), 我们复制了一层, 这些新节点叫它们u',v',... 那么在这个两层的图里, 我们要把原来的这条边变成4条: (u, v), (u, v'), (u', v) 和 (u', v'). 这样就可以保证路径可以经过一个点两次了, 如果想要保证可以经过一个点K次, 只要做K层就好了(每条边变成K^2条).

然后, x也不需要了, 因为每条边的y最大就是1. 但是还有一个问题是目标函数, 由于一条边变成了四条, 如果走四遍的话就多算了四次这条边的长度... 有两种解决办法: 一是类似前面的方法, 给每条边指定定一个indicatrice, 这样就不会重复了, 还有一种更简单的办法是, 在四条边里, 只取一条边的长度保持不变, 其他长度都设成0, 算法自然会优先走那条长度不是0的边(这个没有仔细证明, 不过貌似是这样的). 我们采用了第二个办法....

就这样废了好大功夫, 写好了两层节点的程序, 送去NEOS一跑.... 超时了(超过8h后NEOS会自动停止求解过程)... 后来我们发现即使是不多加新一层的程序, 也会超时. 那节PC里也说, 这种建模方法实际上计算效率比加入2^N个约束的建模还要差...

所以总结一下就是: 问题还是没有解决...

绕过subtour问题

就在一筹莫展之际, Manu提议把搜索的范围缩小: 不用整个巴黎的数据, 只取起点附近的区域, 这样的话, 说不定我们原先的模型得到的结果会少一点connected components... 试了一下, 果然! 虽然没有强制要求路径的连通性, 得到的结果确实(几乎)只有一个联通分支!

后来就想到了一个很聪明的办法: 用八倍的T作为时间限制, 让一辆车去跑, 由于时间很充分, 我们得到的解应该不太会出现多个联通分支, 然后一旦得到了这个路径(后来我们叫它"big path"), 只需要把它分成8段, 每辆车先从起点(巴黎谷歌)跑到每一段的开头, 然后沿着这条路跑就好了啊!! 虽然从巴黎谷歌跑到每一段的起点会稍微浪费一些时间, 但是这点损失其实微不足道!! 而且也不用跑8次LP这么麻烦了 -- 要是早点想到这个就好了!!

而且很神奇的是, 把时间变成了8T以后, 我们的Lp模型计算的更快了: 原先要计算30分钟左右, 现在只要差不多3分钟就出结果了!! 这一点没太想明白, 可能是搜索的空间变小了??

说写就写, 最短路径用Dijkstra算法就能得到了, 不一会就写出了这个把一条big path变成8条small path的程序. 然后我们一举得到了1957596分(离满分还差了两千米左右)!!

IV. 进一步优化

优化的话有两个方向: 第一个方向是修改那个程序代码, 使得每次走到big path某段起点的时候可以少走些路, 不过这个方向应该没什么前途: 程序写起来麻烦不说, 可以改进的空间也很有限, 因为big path一共也没有剩多少时间, 所以八辆车是不可能把big path走完的. 第二个方向则是设法优化big path的时间使用, 使得它在得到最长路径的同时使用尽量少的时间.

引入时间正则项

其实8T作为时间来说非常充裕, 但是我们的解里面, big path还是把时间用的差不多了: 因为时间根本没有出现在目标函数里. 所以, 可以把时间也作为目标函数加上去, 不过要注意最优先要优化的还是路径长度, 所以时间项前面要乘以一个非常小的数(比如0.000001), 类似做regularization.

所以新的目标函数是:

这样一运行, 我们的big path居然剩下了六千多秒的时间!!! 这些时间足够八辆车跑到各段的起点了!! 我们这样, 八辆车走完了big path所有的路程得到的结果离满分只差了7米, 而最后一辆车还有两千多秒没有走!

检查了一下, 发现有一个7米长的路是唯一剩下没走过的路: 于是只要再让最后一辆车去走一下就好了(因为时间够用), 最终我们的方案走完了整个巴黎:

等一下, 这里有点奇怪: 为什么优化得到的结果并不是最优的(没有走完所有的路程)? 我们通过后来手动走那条剩下的路都没有超时, 所以说其实肯定可以在8T的时间走遍所有街道的! 后来查了一下才发现, 原来Gurobi并不是返回最优解, 而是当当前可行解与最优解足够接近的时候就直接停止, 这个参数的名字叫mipgap, 更多参数可以参考这里: http://www.gurobi.com/documentation/6.0/ampl-gurobi-guide/parameters .

这也就解释了为什么NEOS得到的解不是最优的, 不过还好, 通过最后手动添加那条没走的边, 还是走完了所有的街道.

............

只是作为强迫症的话觉得还是有点......不够完美.

直接优化时间

今晚, 机智的Manu想到了解决办法: 既然知道所有的街道都会被走遍, 直接不把它看成目标, 而是直接作为约束好了, 然后约束直接改成优化时间!

要就经过所有街道的约束很简单, 那就是对任何街道e, 都有: y_e+y_e'>=1. 另外肯定所有街道都会经过, x就没有必要存在了.

LP模型表达为:

用这样的模型, 我们的big path 省下了一万两千多秒, 所以我们的最终结果里, 最后的一辆车跑完以后还有9596秒没有用!!

到这一步, 真的可以算是完美解决这个问题了, 而且强迫症也得救了.

Yeah! : )