You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 14:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun May 10, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3770992927,1cyclictest13-21ksoftirqd/000:00:000
3776992621,4cyclictest0-21swapper/223:43:192
3776992523,1cyclictest12062-21ssh22:48:552
377699250,24cyclictest29891-21diskmemload22:57:192
377699250,21cyclictest0-21swapper/222:26:072
3772992524,1cyclictest29891-21diskmemload22:29:171
3770992524,1cyclictest141rcuc/023:52:280
3770992523,1cyclictest0-21swapper/022:28:300
377099250,24cyclictest30821-21sh21:39:000
377099250,24cyclictest0-21swapper/022:08:520
377099250,24cyclictest0-21swapper/000:10:060
3780992422,1cyclictest0-21swapper/323:41:023
3780992422,1cyclictest0-21swapper/323:16:503
378099240,23cyclictest29891-21diskmemload22:01:493
378099240,23cyclictest1168-21ssh22:39:273
378099240,23cyclictest0-21swapper/321:30:403
378099240,23cyclictest0-21swapper/300:06:103
3776992423,1cyclictest0-21swapper/221:15:102
3776992421,0cyclictest0-21swapper/200:18:082
377699240,2cyclictest324-21perf22:10:002
377699240,24cyclictest0-21swapper/200:23:462
377699240,23cyclictest30609-21awk20:05:002
377699240,23cyclictest0-21swapper/223:47:552
377699240,1cyclictest0-21swapper/223:12:502
377299240,23cyclictest7652-21sh23:13:221
377299240,23cyclictest29891-21diskmemload22:13:231
377299240,23cyclictest29891-21diskmemload21:29:591
377299240,23cyclictest23284-21sh22:30:191
377299240,23cyclictest13126-21ssh21:22:531
377299240,0cyclictest29891-21diskmemload22:15:161
377299240,0cyclictest0-21swapper/100:29:051
3770992424,0cyclictest29891-21diskmemload21:15:340
3770992423,1cyclictest22671-21kworker/0:022:18:000
3770992423,1cyclictest13-21ksoftirqd/022:01:440
3770992423,0cyclictest1762-21apt-get00:06:430
3770992422,1cyclictest62550irq/127-eno100:01:460
3770992422,1cyclictest19874-21ssh21:29:150
3770992422,1cyclictest14686-21kworker/u8:221:57:590
3770992421,1cyclictest313-21systemd-journal21:35:000
3770992420,3cyclictest1017-21ssh22:10:440
377099240,23cyclictest30681-21ssh21:10:500
377099240,23cyclictest29891-21diskmemload23:47:370
377099240,23cyclictest29891-21diskmemload23:28:350
377099240,23cyclictest29891-21diskmemload23:10:040
377099240,23cyclictest29891-21diskmemload23:06:410
377099240,23cyclictest27518-21perf21:05:010
377099240,23cyclictest0-21swapper/023:04:160
377099240,23cyclictest0-21swapper/022:41:290
377099240,23cyclictest0-21swapper/019:51:440
3780992323,0cyclictest29891-21diskmemload22:13:053
3780992323,0cyclictest0-21swapper/323:45:253
3780992323,0cyclictest0-21swapper/323:01:363
3780992323,0cyclictest0-21swapper/322:27:323
3780992323,0cyclictest0-21swapper/300:30:063
3780992322,1cyclictest0-21swapper/319:31:433
3780992321,1cyclictest18276-21sh00:21:193
3780992321,1cyclictest1098-21sh23:07:003
3780992321,1cyclictest0-21swapper/323:27:253
3780992321,1cyclictest0-21swapper/322:06:473
3780992321,1cyclictest0-21swapper/321:45:023
3780992321,1cyclictest0-21swapper/321:17:153
378099230,23cyclictest0-21swapper/321:52:313
378099230,23cyclictest0-21swapper/300:16:343
378099230,22cyclictest7444-21ssh00:11:453
378099230,22cyclictest28532-21/usr/sbin/munin00:01:543
378099230,22cyclictest26769-21sh23:59:383
378099230,22cyclictest2139-21ssh21:13:513
378099230,22cyclictest13170-21rm22:50:113
378099230,22cyclictest10601-21apt-get21:21:423
378099230,22cyclictest0-21swapper/321:56:083
378099230,1cyclictest0-21swapper/322:33:113
378099230,0cyclictest0-21swapper/323:54:553
378099230,0cyclictest0-21swapper/323:36:463
378099230,0cyclictest0-21swapper/323:14:483
378099230,0cyclictest0-21swapper/321:38:183
3776992323,0cyclictest29891-21diskmemload23:20:222
3776992323,0cyclictest29891-21diskmemload21:30:182
3776992323,0cyclictest0-21swapper/223:17:062
3776992323,0cyclictest0-21swapper/221:56:032
3776992323,0cyclictest0-21swapper/221:50:422
3776992323,0cyclictest0-21swapper/200:25:382
3776992323,0cyclictest0-21swapper/200:01:252
3776992321,1cyclictest0-21swapper/223:52:352
3776992320,2cyclictest0-21swapper/222:40:422
377699230,2cyclictest32223-21rm21:40:382
377699230,2cyclictest30328-21rm23:33:312
377699230,2cyclictest20374-21latency_hist21:30:002
377699230,2cyclictest1179-21ssh22:10:562
377699230,2cyclictest10001-21ssh00:12:582
377699230,23cyclictest0-21swapper/223:39:492
377699230,22cyclictest20650-21apt-get20:51:452
377699230,0cyclictest0-21swapper/223:57:292
377699230,0cyclictest0-21swapper/223:07:402
377699230,0cyclictest0-21swapper/221:23:012
377299234,0cyclictest783-21snmpd21:11:261
3772992323,0cyclictest0-21swapper/123:45:441
3772992323,0cyclictest0-21swapper/122:08:311
3772992323,0cyclictest0-21swapper/100:01:401
3772992322,1cyclictest29891-21diskmemload21:41:521
3772992322,1cyclictest22958-21ssh22:01:421
3772992321,1cyclictest29893-21ssh21:10:011
3772992321,1cyclictest2408-21ssh23:36:571
3772992321,1cyclictest16289-21rm23:50:141
3772992321,1cyclictest0-21swapper/122:41:541
377299231,1cyclictest25118-21lockfile-create21:00:011
377299230,2cyclictest8134-21sh22:45:531
377299230,2cyclictest19956-21ssh23:24:451
377299230,23cyclictest0-21swapper/123:01:201
377299230,23cyclictest0-21swapper/121:15:131
377299230,22cyclictest6532-21sh00:38:241
377299230,22cyclictest21490-21ssh23:26:321
377299230,22cyclictest0-21swapper/122:20:181
377299230,22cyclictest0-21swapper/121:30:531
377299230,22cyclictest0-21swapper/119:21:441
377299230,1cyclictest313-21systemd-journal20:54:561
377299230,1cyclictest0-21swapper/122:00:011
377299230,0cyclictest29891-21diskmemload21:50:381
377299230,0cyclictest14357-21apt-get22:51:451
377299230,0cyclictest0-21swapper/123:17:561
377299230,0cyclictest0-21swapper/122:55:451
377299230,0cyclictest0-21swapper/121:39:091
377299230,0cyclictest0-21swapper/100:34:181
3770992323,0cyclictest0-21swapper/023:40:310
3770992323,0cyclictest0-21swapper/022:57:080
377099231,1cyclictest0-21swapper/022:45:410
377099230,2cyclictest18588-21ssh23:22:500
377099230,23cyclictest0-21swapper/000:34:100
377099230,22cyclictest32026-21rm22:37:160
377099230,22cyclictest29891-21diskmemload21:45:390
377099230,22cyclictest12620-21sh21:22:100
377099230,22cyclictest0-21swapper/000:18:490
377099230,21cyclictest0-21swapper/022:53:290
377099230,1cyclictest11352-21sh21:51:100
377099230,0cyclictest29891-21diskmemload21:42:590
377099230,0cyclictest0-21swapper/022:30:570
377099230,0cyclictest0-21swapper/000:25:150
3780992222,0cyclictest0-21swapper/321:25:073
3780992222,0cyclictest0-21swapper/300:28:363
3780992220,1cyclictest30996-21apt-get20:06:433
3780992220,1cyclictest20633-21awk19:45:003
3780992220,1cyclictest0-21swapper/323:20:153
3780992220,1cyclictest0-21swapper/322:45:543
3780992220,1cyclictest0-21swapper/300:35:253
378099220,22cyclictest0-21swapper/323:30:063
378099220,22cyclictest0-21swapper/322:20:293
378099220,21cyclictest23397-21apt-get19:51:443
378099220,21cyclictest0-21swapper/322:41:343
378099220,21cyclictest0-21swapper/322:16:073
378099220,1cyclictest3003-21perf20:15:013
378099220,1cyclictest0-21swapper/321:41:143
3776992222,0cyclictest29891-21diskmemload00:05:572
3776992222,0cyclictest0-21swapper/223:25:372
3776992222,0cyclictest0-21swapper/221:36:172
3776992221,1cyclictest0-21swapper/221:45:552
3776992220,1cyclictest6528-21apt-get19:16:432
3776992219,2cyclictest6193-21rm22:15:012
377699220,22cyclictest0-21swapper/222:20:042
377699220,21cyclictest0-21swapper/222:35:292
377699220,21cyclictest0-21swapper/222:01:352
377699220,1cyclictest17125-21sh22:53:092
377699220,1cyclictest0-21swapper/221:12:572
377699220,1cyclictest0-21swapper/200:33:112
377699220,0cyclictest0-21swapper/223:02:542
377699220,0cyclictest0-21swapper/222:30:142
3772992222,0cyclictest0-21swapper/122:38:541
3772992220,1cyclictest0-21swapper/100:20:171
3772992219,2cyclictest701-21ssh00:05:211
377299221,21cyclictest0-21swapper/123:41:061
377299220,21cyclictest26409-21ssh23:30:381
377299220,21cyclictest22951-21apt-get23:56:451
377299220,21cyclictest0-21swapper/121:46:321
377299220,21cyclictest0-21swapper/120:40:001
377299220,21cyclictest0-21swapper/119:55:021
377299220,21cyclictest0-21swapper/100:11:431
377299220,1cyclictest12960-21ssh00:16:451
3770992222,0cyclictest29891-21diskmemload00:21:130
3770992222,0cyclictest0-21swapper/023:16:020
3770992221,1cyclictest0-21swapper/023:31:310
3770992221,0cyclictest0-21swapper/020:51:450
3770992220,1cyclictest2345-21apt-get00:35:020
3770992220,1cyclictest0-21swapper/020:20:010
377099220,21cyclictest554-21latency_hist20:10:010
377099220,21cyclictest0-21swapper/023:35:210
377099220,1cyclictest26187-21apt-get19:56:420
3780992120,1cyclictest16134-21apt-get19:36:433
378099210,1cyclictest26026-21apt-get19:55:023
3776992121,0cyclictest10991-21apt-get20:31:432
3776992119,1cyclictest0-21swapper/220:25:012
377699210,20cyclictest26187-21apt-get19:56:422
377699210,20cyclictest13739-21apt-get19:31:442
377699210,0cyclictest0-21swapper/220:25:022
3772992121,0cyclictest0-21swapper/121:01:451
3772992121,0cyclictest0-21swapper/120:50:001
3772992120,1cyclictest30996-21apt-get20:06:431
3772992119,1cyclictest18204-21sendmail19:40:011
377299210,21cyclictest0-21swapper/120:25:021
377299210,21cyclictest0-21swapper/120:11:441
377299210,0cyclictest0-21swapper/119:41:431
3770992120,0cyclictest0-21swapper/019:21:440
377099210,20cyclictest11341-21apt-get19:26:430
377099210,20cyclictest0-21swapper/019:46:430
377099210,1cyclictest6528-21apt-get19:16:430
377099210,0cyclictest0-21swapper/020:46:440
377099210,0cyclictest0-21swapper/020:41:450
377099210,0cyclictest0-21swapper/020:21:440
3780992020,0cyclictest0-21swapper/321:01:453
3780992020,0cyclictest0-21swapper/320:41:453
378099200,0cyclictest8594-21apt-get20:26:443
378099200,0cyclictest0-21swapper/319:16:433
3776992019,1cyclictest25512-21apt-get21:01:442
377699200,20cyclictest0-21swapper/220:16:432
377699200,20cyclictest0-21swapper/219:26:432
377699200,19cyclictest27904-21apt-get21:06:442
377699200,0cyclictest0-21swapper/219:41:432
3772992020,0cyclictest0-21swapper/120:01:421
3772992020,0cyclictest0-21swapper/119:26:431
3772992020,0cyclictest0-21swapper/119:16:431
377299200,20cyclictest0-21swapper/120:41:451
377299200,20cyclictest0-21swapper/120:21:441
377299200,0cyclictest0-21swapper/119:31:431
377099200,20cyclictest0-21swapper/020:11:440
3780991919,0cyclictest0-21swapper/321:06:443
3780991919,0cyclictest0-21swapper/320:31:433
3780991919,0cyclictest0-21swapper/320:21:443
3780991919,0cyclictest0-21swapper/319:11:443
3776991919,0cyclictest0-21swapper/220:56:452
3776991919,0cyclictest0-21swapper/219:46:432
3776991919,0cyclictest0-21swapper/219:11:442
3770991919,0cyclictest0-21swapper/020:31:440
3770991919,0cyclictest0-21swapper/020:01:430
3780991818,0cyclictest0-21swapper/319:26:433
3780991817,0cyclictest3373-21apt-get20:16:433
3776991818,0cyclictest0-21swapper/220:06:432
3776991817,1cyclictest18254-21apt-get20:46:442
3772991818,0cyclictest0-21swapper/119:51:441
3772991817,0cyclictest4134-21apt-get19:11:441
3772991816,1cyclictest0-21swapper/120:35:001
3770991818,0cyclictest0-21swapper/020:56:450
3770991818,0cyclictest0-21swapper/019:41:440
3770991817,1cyclictest0-21swapper/020:30:000
3780991717,0cyclictest0-21swapper/319:21:443
3780991714,2cyclictest30599-21awk20:05:003
3772991715,1cyclictest23024-21munin-run19:50:001
3770991717,0cyclictest0-21swapper/020:36:440
3770991717,0cyclictest0-21swapper/019:36:430
3770991716,1cyclictest27904-21apt-get21:06:440
377099170,2cyclictest15796-21sed19:35:010
3780991615,0cyclictest0-21swapper/320:46:443
3780991614,1cyclictest25144-21sort21:00:013
3780991515,0cyclictest0-21swapper/320:51:453
3776991515,0cyclictest0-21swapper/220:36:442
3776991515,0cyclictest0-21swapper/220:11:442
3776991514,0cyclictest0-21swapper/219:21:442
377699150,9cyclictest0-21swapper/219:51:442
3770991515,0cyclictest0-21swapper/019:11:440
3780991414,0cyclictest0-21swapper/319:45:103
378099140,14cyclictest0-21swapper/320:35:543
3776991414,0cyclictest0-21swapper/219:35:242
377699140,14cyclictest0-21swapper/220:40:262
377299140,0cyclictest0-21swapper/120:15:071
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional