You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-19 - 04:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Thu Mar 19, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12530992623,2cyclictest32020-21ssh21:34:181
1253899250,24cyclictest331rcuc/321:20:473
1253899250,24cyclictest0-21swapper/322:02:453
1253899250,24cyclictest0-21swapper/321:34:183
12533992523,1cyclictest0-21swapper/222:35:012
12530992525,0cyclictest6522-21diskmemload21:46:211
12530992522,2cyclictest29687-21ssh22:02:451
1252599250,24cyclictest6522-21diskmemload22:29:500
1252599250,24cyclictest6340-21ssh21:41:240
12538992423,1cyclictest6522-21diskmemload22:54:053
12538992422,1cyclictest0-21swapper/323:39:233
1253899240,23cyclictest6522-21diskmemload23:10:213
1253899240,23cyclictest0-21swapper/322:41:053
1253899240,0cyclictest6522-21diskmemload22:12:123
12533992423,1cyclictest30289-21ssh22:03:492
12533992422,1cyclictest23982-21ssh21:27:212
1253399241,0cyclictest271rcuc/223:29:432
1253399240,23cyclictest15903-21apt-get22:22:212
1253399240,23cyclictest1391-21ssh23:41:492
1253399240,23cyclictest10393-21ssh23:17:512
1253399240,23cyclictest0-21swapper/222:50:452
1253399240,0cyclictest6522-21diskmemload23:59:502
1253099240,24cyclictest0-21swapper/121:58:281
1253099240,24cyclictest0-21swapper/121:40:491
1253099240,24cyclictest0-21swapper/100:33:521
1253099240,21cyclictest0-21swapper/121:25:381
1253099240,1cyclictest6110-21ssh23:45:331
1253099240,1cyclictest23140-21sh22:59:291
1253099240,0cyclictest6522-21diskmemload23:44:291
12525992423,1cyclictest18691-21ssh00:28:280
12525992422,2cyclictest13331-21ssh23:22:060
12525992421,2cyclictest14018-21rm22:51:370
1252599240,2cyclictest7708-21ssh23:16:320
1252599240,24cyclictest6522-21diskmemload21:59:500
1252599240,23cyclictest17608-21ssh21:20:340
1252599240,23cyclictest0-21swapper/021:51:580
1252599240,0cyclictest6522-21diskmemload21:17:490
1252599240,0cyclictest6522-21diskmemload00:31:430
1252599240,0cyclictest0-21swapper/022:05:530
12538992323,0cyclictest0-21swapper/322:06:373
12538992323,0cyclictest0-21swapper/321:41:313
12538992323,0cyclictest0-21swapper/321:26:313
12538992323,0cyclictest0-21swapper/300:25:133
12538992322,1cyclictest0-21swapper/322:59:323
12538992321,1cyclictest0-21swapper/323:26:173
12538992321,1cyclictest0-21swapper/322:39:403
12538992321,1cyclictest0-21swapper/321:54:063
12538992321,1cyclictest0-21swapper/321:48:573
12538992321,1cyclictest0-21swapper/300:32:233
12538992321,1cyclictest0-21swapper/300:10:333
1253899231,1cyclictest34-21ksoftirqd/300:00:533
1253899230,2cyclictest14511-21cron20:20:003
1253899230,23cyclictest6522-21diskmemload21:15:493
1253899230,23cyclictest0-21swapper/323:50:143
1253899230,23cyclictest0-21swapper/321:39:473
1253899230,22cyclictest5133-21turbostat.cron23:45:003
1253899230,22cyclictest26490-21apt-get00:37:213
1253899230,22cyclictest24899-21sh21:58:273
1253899230,22cyclictest13406-21ssh00:23:123
1253899230,22cyclictest0-21swapper/322:30:423
1253899230,1cyclictest6522-21diskmemload22:26:143
1253899230,0cyclictest6522-21diskmemload23:57:223
1253899230,0cyclictest0-21swapper/322:23:213
12533992323,0cyclictest6522-21diskmemload23:03:342
12533992323,0cyclictest0-21swapper/223:21:252
12533992323,0cyclictest0-21swapper/222:45:122
12533992323,0cyclictest0-21swapper/221:36:282
12533992322,1cyclictest6522-21diskmemload22:17:092
12533992321,1cyclictest0-21swapper/223:31:112
12533992320,0cyclictest0-21swapper/222:09:292
12533992320,0cyclictest0-21swapper/221:15:112
1253399230,2cyclictest29240-21ssh23:05:292
1253399230,23cyclictest17637-21kworker/u8:021:33:422
1253399230,23cyclictest0-21swapper/221:59:542
1253399230,23cyclictest0-21swapper/221:50:272
1253399230,23cyclictest0-21swapper/221:11:252
1253399230,22cyclictest6522-21diskmemload23:35:012
1253399230,22cyclictest6522-21diskmemload00:24:112
1253399230,22cyclictest6095-21ssh22:42:472
1253399230,22cyclictest313-21systemd-journal19:50:002
1253399230,22cyclictest28-21ksoftirqd/200:30:482
1253399230,22cyclictest22498-21ssh22:58:312
1253399230,22cyclictest21099-21ssh22:27:212
1253399230,22cyclictest1753-21sendmail-msp21:00:012
1253399230,22cyclictest0-21swapper/200:19:302
1253399230,0cyclictest271rcuc/221:23:252
1253399230,0cyclictest0-21swapper/222:38:572
1253399230,0cyclictest0-21swapper/222:10:562
1253399230,0cyclictest0-21swapper/200:14:042
12530992323,0cyclictest0-21swapper/122:46:481
12530992323,0cyclictest0-21swapper/122:22:041
12530992323,0cyclictest0-21swapper/122:17:051
12530992323,0cyclictest0-21swapper/122:05:131
12530992323,0cyclictest0-21swapper/100:03:231
12530992321,1cyclictest0-21swapper/122:12:321
1253099230,23cyclictest0-21swapper/123:18:091
1253099230,23cyclictest0-21swapper/121:20:571
1253099230,23cyclictest0-21swapper/100:27:241
1253099230,22cyclictest6522-21diskmemload22:36:161
1253099230,22cyclictest5983-21ssh22:42:371
1253099230,22cyclictest3914-21ssh21:38:041
1253099230,22cyclictest23398-21ssh22:28:181
1253099230,22cyclictest22180-21apt-get20:37:201
1253099230,22cyclictest21746-21ls20:35:001
1253099230,22cyclictest17180-21ssh23:56:381
1253099230,22cyclictest14521-21sh20:20:001
1253099230,22cyclictest14081-21ssh00:24:261
1253099230,22cyclictest10500-21sh21:13:001
1253099230,21cyclictest27335-21sh23:02:521
1253099230,0cyclictest6522-21diskmemload23:11:151
1253099230,0cyclictest17390-21apt-get20:27:221
1253099230,0cyclictest0-21swapper/123:53:591
1253099230,0cyclictest0-21swapper/100:14:391
12525992323,0cyclictest6522-21diskmemload23:07:350
12525992323,0cyclictest6522-21diskmemload21:11:020
12525992323,0cyclictest6522-21diskmemload00:16:380
12525992323,0cyclictest6522-21diskmemload00:10:190
12525992323,0cyclictest0-21swapper/022:20:580
12525992323,0cyclictest0-21swapper/022:16:470
12525992321,1cyclictest5793-21ssh23:45:050
12525992321,1cyclictest24158-21turbostat20:40:000
12525992321,1cyclictest11434-21ssh21:46:120
12525992321,1cyclictest0-21swapper/022:57:400
12525992319,2cyclictest6551-21munin-plugin-st21:10:000
1252599230,2cyclictest5344-21latency_hist21:40:010
1252599230,2cyclictest30191-21awk21:32:270
1252599230,2cyclictest22133-21latency_hist23:30:010
1252599230,2cyclictest2021-21ssh23:10:320
1252599230,23cyclictest0-21swapper/022:38:420
1252599230,22cyclictest26963-21ssh22:01:170
1252599230,22cyclictest26173-21diskmemload21:28:120
1252599230,22cyclictest22871-21sh23:30:510
1252599230,22cyclictest11856-21ssh22:48:360
1252599230,0cyclictest0-21swapper/022:31:180
1252599230,0cyclictest0-21swapper/022:13:550
1252599230,0cyclictest0-21swapper/000:05:350
12538992222,0cyclictest0-21swapper/323:46:213
12538992222,0cyclictest0-21swapper/323:15:343
12538992221,1cyclictest6522-21diskmemload22:46:203
12538992221,1cyclictest0-21swapper/300:15:213
12538992220,1cyclictest9503-21rm22:15:223
12538992220,1cyclictest0-21swapper/323:05:343
1253899220,2cyclictest6522-21diskmemload21:10:113
1253899220,21cyclictest0-21swapper/323:02:333
1253899220,21cyclictest0-21swapper/319:22:203
1253899220,21cyclictest0-21swapper/300:07:373
1253899220,1cyclictest0-21swapper/323:20:143
12533992222,0cyclictest0-21swapper/223:11:502
12533992222,0cyclictest0-21swapper/200:27:342
12533992220,1cyclictest6522-21diskmemload23:51:042
12533992220,1cyclictest16138-21kworker/2:123:45:022
1253399220,21cyclictest6522-21diskmemload00:35:542
1253399220,21cyclictest6522-21diskmemload00:05:062
1253399220,21cyclictest5844-21ssh21:40:392
1253399220,0cyclictest6522-21diskmemload21:45:542
12530992222,0cyclictest0-21swapper/123:26:061
12530992222,0cyclictest0-21swapper/123:05:071
12530992222,0cyclictest0-21swapper/122:51:041
12530992222,0cyclictest0-21swapper/100:37:151
1253099220,22cyclictest12959-21rm21:16:231
1253099220,22cyclictest0-21swapper/122:31:441
1253099220,21cyclictest0-21swapper/100:15:211
1253099220,1cyclictest20920-21kworker/u8:023:35:341
1253099220,1cyclictest13113-21apt-get19:10:021
1253099220,0cyclictest6522-21diskmemload21:50:481
12525992222,0cyclictest0-21swapper/023:36:110
12525992222,0cyclictest0-21swapper/000:35:480
12525992220,1cyclictest10964-21rm23:50:050
1252599220,2cyclictest4331-21ssh23:43:440
1252599220,22cyclictest3296-21sh22:41:030
1252599220,21cyclictest15723-21apt-get19:17:210
1252599220,1cyclictest32181-21apt-get20:57:220
1252599220,1cyclictest22937-21apt-get00:02:210
1252599220,0cyclictest0-21swapper/023:56:070
12538992121,0cyclictest0-21swapper/323:30:033
12538992120,1cyclictest0-21swapper/321:02:223
1253899210,1cyclictest0-21swapper/319:12:203
1253899210,0cyclictest0-21swapper/319:37:213
1253899210,0cyclictest0-21swapper/319:27:203
12533992121,0cyclictest0-21swapper/220:32:182
1253399210,20cyclictest22934-21apt-get19:32:212
1253399210,20cyclictest0-21swapper/220:52:212
1253099210,21cyclictest0-21swapper/120:52:211
1253099210,21cyclictest0-21swapper/119:22:201
1253099210,0cyclictest25722-21apt-get19:37:211
1253099210,0cyclictest24597-21apt-get20:42:221
1253099210,0cyclictest0-21swapper/120:22:191
1253099210,0cyclictest0-21swapper/100:05:351
12525992121,0cyclictest0-21swapper/019:37:210
12525992121,0cyclictest0-21swapper/000:20:290
1252599210,20cyclictest2185-21apt-get21:02:220
1252599210,20cyclictest0-21swapper/019:52:230
1252599210,1cyclictest12441-21apt-get19:10:020
1252599210,0cyclictest7728-21apt-get20:07:210
1252599210,0cyclictest0-21swapper/020:32:180
12538992020,0cyclictest0-21swapper/321:07:213
12538992020,0cyclictest0-21swapper/319:32:213
1253899200,0cyclictest28142-21apt-get19:42:223
1253899200,0cyclictest0-21swapper/320:42:213
1253899200,0cyclictest0-21swapper/319:57:213
1253899200,0cyclictest0-21swapper/319:52:233
12533992020,0cyclictest0-21swapper/220:17:212
12533992020,0cyclictest0-21swapper/220:07:212
12533992019,0cyclictest0-21swapper/219:57:212
1253399200,0cyclictest0-21swapper/220:27:222
12530992020,0cyclictest2917-21apt-get19:57:211
12530992020,0cyclictest0-21swapper/119:42:221
12525992020,0cyclictest0-21swapper/020:42:210
1252599200,0cyclictest0-21swapper/020:47:220
12538991917,1cyclictest0-21swapper/320:15:013
12533991918,1cyclictest0-21swapper/219:17:212
12530991919,0cyclictest4579-21apt-get21:07:211
12530991919,0cyclictest0-21swapper/119:27:201
12530991918,1cyclictest30534-21apt-get19:47:211
12525991919,0cyclictest14993-21apt-get20:22:190
12525991919,0cyclictest0-21swapper/020:27:220
12525991919,0cyclictest0-21swapper/020:17:210
12538991818,0cyclictest0-21swapper/319:17:213
1253899180,17cyclictest29029-21apt-config20:50:013
12533991818,0cyclictest0-21swapper/220:42:212
12533991818,0cyclictest0-21swapper/220:37:212
12533991818,0cyclictest0-21swapper/220:12:212
12533991818,0cyclictest0-21swapper/219:35:022
12533991818,0cyclictest0-21swapper/219:22:212
12533991816,1cyclictest6545-21perf21:10:002
12533991816,1cyclictest26992-21apt-get20:47:222
1253399181,3cyclictest28-21ksoftirqd/220:01:082
12530991818,0cyclictest0-21swapper/119:52:231
12530991817,0cyclictest0-21swapper/121:02:221
12538991717,0cyclictest0-21swapper/320:37:203
12538991717,0cyclictest0-21swapper/320:32:183
12538991716,1cyclictest5335-21apt-get20:02:213
1253899170,14cyclictest0-21swapper/320:20:443
12533991717,0cyclictest0-21swapper/220:22:192
12530991716,0cyclictest0-21swapper/119:32:211
12530991714,2cyclictest29018-21munin-run20:50:011
12525991717,0cyclictest0-21swapper/019:47:210
12538991616,0cyclictest0-21swapper/320:50:023
12533991614,1cyclictest22488-21perf19:30:002
12533991614,1cyclictest0-21swapper/221:02:222
12525991616,0cyclictest0-21swapper/020:12:210
12525991616,0cyclictest0-21swapper/020:02:210
1252599160,16cyclictest0-21swapper/020:52:060
12538991515,0cyclictest0-21swapper/320:57:223
1253899150,0cyclictest0-21swapper/319:47:213
12530991515,0cyclictest9744-21perf20:10:011
12530991515,0cyclictest0-21swapper/119:17:211
12525991515,0cyclictest0-21swapper/019:57:210
12525991513,1cyclictest24906-21munin-run19:35:000
1252599150,14cyclictest20957-21diskstats19:27:220
1253899140,14cyclictest0-21swapper/320:05:063
1253899140,0cyclictest0-21swapper/320:25:033
1253399140,0cyclictest0-21swapper/219:50:032
1253399140,0cyclictest0-21swapper/219:41:212
12530991414,0cyclictest0-21swapper/120:00:121
12525991414,0cyclictest0-21swapper/019:40:180
12525991414,0cyclictest0-21swapper/019:20:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional