You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 04:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack3slot3s.osadl.org (updated Wed Nov 26, 2025 00:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
127572212956,68sleep30-21swapper/319:09:003
12747728454,9sleep10-21swapper/119:07:451
12734828152,9sleep20-21swapper/219:06:102
12751628052,19sleep00-21swapper/019:08:160
127780993530,2cyclictest0-21swapper/323:25:143
5784022272,17sleep1574452-21/usr/sbin/munin00:30:191
127780992724,1cyclictest0-21swapper/322:50:193
127780992724,1cyclictest0-21swapper/320:35:003
127780992521,2cyclictest0-21swapper/323:45:213
127780992515,5cyclictest2968862sleep321:33:533
12778099251,18cyclictest187868-21taskset20:00:253
12778099251,18cyclictest187868-21taskset20:00:253
127780992418,3cyclictest0-21swapper/322:22:293
127780992411,9cyclictest288887-21taskset21:28:473
12778099232,13cyclictest424614-21taskset22:55:063
12778099231,11cyclictest581061-21taskset00:34:043
127780992310,10cyclictest321074-21turbostat21:54:553
4058832221,3sleep10-21swapper/122:41:271
12778099222,15cyclictest589029-21turbostat00:39:593
127780992214,4cyclictest4085162sleep322:45:093
12778099221,16cyclictest460559-21chrt23:15:233
12778099212,13cyclictest570984-21taskset00:25:183
12778099212,13cyclictest397800-21taskset22:36:423
12778099211,16cyclictest0-21swapper/320:35:143
12778099211,15cyclictest556774-21chrt00:15:313
12778099211,15cyclictest384633-21chrt22:30:123
5095482205,2sleep1321rcuc/123:46:321
12778099202,13cyclictest438126-21taskset23:01:323
12778099202,13cyclictest438126-21taskset23:01:313
12778099202,13cyclictest280631-21taskset21:23:143
127780992016,2cyclictest0-21swapper/322:40:163
127780992016,2cyclictest0-21swapper/319:44:593
12778099201,15cyclictest0-21swapper/319:25:213
12778099201,14cyclictest0-21swapper/300:21:163
5888832191,7sleep2257700-21diskmemload00:39:492
5785492191,5sleep2578597-21ntpq00:30:202
12778099192,12cyclictest542844-21taskset00:08:263
12778099192,12cyclictest343123-21taskset22:02:183
12778099192,12cyclictest343123-21taskset22:02:183
12778099192,12cyclictest318554-21taskset21:45:433
127780991916,1cyclictest0-21swapper/322:05:283
127780991916,1cyclictest0-21swapper/321:15:123
127780991916,1cyclictest0-21swapper/320:55:173
127780991916,1cyclictest0-21swapper/319:30:243
127780991915,2cyclictest0-21swapper/322:15:163
127780991914,3cyclictest66550irq/131-enp1s0-TxRx-323:30:113
12778099191,14cyclictest0-21swapper/323:20:303
12778099191,13cyclictest452881-21chrt23:10:253
12778099191,13cyclictest264287-21taskset21:12:393
5290042181,12sleep1530555-21tr00:00:201
4391642181,4sleep2257700-21diskmemload23:03:212
4391642181,4sleep2257700-21diskmemload23:03:202
3125842181,5sleep2257700-21diskmemload21:43:552
12778099189,6cyclictest173059-21taskset19:50:113
12778099186,9cyclictest66550irq/131-enp1s0-TxRx-322:10:283
127780991815,1cyclictest0-21swapper/322:25:153
127780991815,1cyclictest0-21swapper/320:40:163
127780991815,1cyclictest0-21swapper/319:55:133
127780991813,3cyclictest0-21swapper/300:00:203
12778099181,15cyclictest0-21swapper/323:43:063
12778099181,14cyclictest446334-21chrt23:07:133
12778099181,13cyclictest0-21swapper/323:55:133
12778099181,13cyclictest0-21swapper/321:36:523
12778099181,13cyclictest0-21swapper/300:10:043
12778099180,15cyclictest204367-21taskset20:19:343
5733372171,10sleep2257700-21diskmemload00:28:262
5036882171,4sleep1257700-21diskmemload23:45:001
4144742171,2sleep00-21swapper/022:47:100
3821832171,9sleep1257700-21diskmemload22:27:031
3813352171,11sleep2381345-21ssh22:25:392
3591462171,4sleep0257700-21diskmemload22:12:560
3523772171,2sleep00-21swapper/022:09:420
3438852171,4sleep1257700-21diskmemload22:03:391
3438852171,4sleep1257700-21diskmemload22:03:381
3271552171,3sleep20-21swapper/221:52:012
2735652171,10sleep2257700-21diskmemload21:19:492
12778099177,7cyclictest247397-21chrt21:00:013
127780991713,2cyclictest0-21swapper/320:50:173
12778099171,12cyclictest493275-21taskset23:35:563
12778099171,12cyclictest311376-21taskset21:41:503
12778099171,11cyclictest133045-21sleep19:15:003
12778099170,14cyclictest0-21swapper/323:50:063
5514182161,10sleep2257700-21diskmemload00:14:482
5262412161,10sleep2526248-21ssh23:57:392
4405422161,10sleep0440747-21cstates23:05:140
3906812160,2sleep20-21swapper/222:33:062
3674992160,3sleep10-21swapper/122:18:491
3667692160,2sleep00-21swapper/022:17:320
3432222161,3sleep0257700-21diskmemload22:02:290
3432222161,3sleep0257700-21diskmemload22:02:280
3028182161,2sleep10-21swapper/121:35:431
2968962161,4sleep0296909-21ssh21:33:550
12778099161,9cyclictest136016-21taskset19:15:123
12778099161,3cyclictest0-21swapper/321:55:213
12778099161,3cyclictest0-21swapper/321:55:213
127780991613,1cyclictest0-21swapper/320:05:153
127780991612,2cyclictest0-21swapper/321:05:243
12778099161,11cyclictest172816-21turbostat19:49:593
12778099161,10cyclictest204683-21chrt20:20:093
12778099160,11cyclictest194143-21taskset20:10:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional