老博客 on Code Now

EM Algorithm Notes

Tue, 14 Apr 2020 17:57:00 +0000

I recently ran into the EM algorithm again. Too often I just use something without knowing why, so this time, before using it, I went back and read up on it some more.

Where the EM algorithm applies: maximum likelihood estimation of the parameters of a probabilistic model that has hidden variables.

First question: what is a hidden variable?

Latent Variables

Something that cannot be directly observed, but that influences the state of the system and the outputs that can be observed. It refers to an unobservable random variable. Latent variables can be inferred from observed data by using a mathematical model.

Python Basics: Decorators

Fri, 20 Mar 2020 16:54:00 +0000

A decorator is a callable object whose argument is another function (the decorated function). The decorator may process the decorated function and then return it, or replace it with another function or callable object.

1@test
2def target():
3 print("run target function!")

It works the same as the following code

1def target():
2 print("run target function!")
3target = test(target) # In the test function, the function will be enhanced or replaced

Function decorators run at import time, while the decorated function only runs when it is explicitly called.

[Python Basics] Namespaces, Scope, and Closures

Fri, 20 Mar 2020 16:53:00 +0000

A namespace is a mapping from names to objects. Most namespaces are currently implemented as Python dictionaries, but that’s normally not noticeable in any way (except for performance), and it may change in the future.

Namespaces are an important structure for avoiding name collisions. Namespaces are dynamic: they come into being as the interpreter executes, and they are maintained throughout the life of the program.

Namespaces:

Built-in namespace: the namespace that comes with the Python interpreter as soon as it starts, and stops when the Python interpreter ends.
Global namespace: the set of global names of a Python module — names defined directly in the module, such as classes, functions, other imported modules, and so on. Destroyed when the interpreter exits.
Local namespace: the namespace inside a function or a class. It is created when a function is called, and deleted when the function returns or raises an error that is not handled inside the function.
The attribute namespace of an object
Class namespace

Scope

A scope is a textual region of a Python program where a namespace is directly accessible. “Directly accessible” here means that an unqualified reference to a name attempts to find the name in the namespace.

Inversions in an array

Tue, 10 Mar 2020 18:11:00 +0000

Interview Question 51. Inversions in an Array Given two numbers in an array, if the earlier number is greater than the later one, the two numbers form an inversion. Given an array, find the total number of inversions in it.

Example 1:

Input: [7,5,6,4]
Output: 5

Constraints: 0 <= array length <= 50000

Approach

Brute-force solution

Just scan it mindlessly

 1class Solution {
 2public:
 3 int reversePairs(vector<int>& nums) {
 4 if (nums.size() == 0)
 5 return 0;
 6 int sum = 0;
 7 for (int i = 0; i < nums.size() - 1; ++i) {
 8 for (int j = i+1; j < nums.size(); ++j) {
 9 if (nums[i] > nums[j])
10 sum++;
11 }
12 }
13 return sum;
14 }
15};

Time complexity $O(n^2)$, space complexity $O(1)$

Fenwick Tree (Binary Indexed)

Tue, 10 Mar 2020 17:30:00 +0000

First, many thanks to the Bilibili creator 鹤翔万里 for the video; I recommend watching https://www.bilibili.com/video/av69667943?from=search&seid=10916758362943551299. This article is a summary I wrote for the 算法笔记 collection accompanying that video.

Problem setup: given an array of length n, support the following two operations

Output the sum of every number in the interval [x, y]
Add k to the x-th number

The most basic algorithm:

Maintain an array sum, where sum[i] stores the sum from 0 to i. Its recurrence relation is sum[i] = sum[i-1] + nums[i], where sum[0] = nums[0].
The interval sum can then be computed using sum_xy = sum[y] - sum[x-1].
To add v to the x-th number, since all values of sum[x]...sum[n] must be updated, the time complexity is $O(n)$
If k operations are performed, add, then the time complexity is $O(kn)$, and the time complexity of k interval-sum queries is $O(k)$

In many scenarios with frequent updates, the $O(kn)$ time complexity is unacceptable.