You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-03 - 14:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Fri Apr 03, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6125992724,2cyclictest13317-21cron19:25:013
612599260,25cyclictest20813-21sh21:30:353
611499260,25cyclictest1039-21cstates23:18:391
612599250,24cyclictest23848-21sh22:37:353
612599250,24cyclictest0-21swapper/323:08:443
6122992520,2cyclictest4766-21rm21:14:522
6122992518,6cyclictest0-21swapper/221:24:402
612299251,23cyclictest7498-21rm23:23:452
612299250,24cyclictest0-21swapper/223:29:442
6114992523,1cyclictest0-21swapper/123:33:471
611499250,1cyclictest0-21swapper/122:52:581
611199253,1cyclictest62550irq/127-eno122:27:160
611199250,3cyclictest15081-21sh21:25:000
611199250,24cyclictest0-21swapper/023:23:000
611199250,24cyclictest0-21swapper/022:45:500
611199250,24cyclictest0-21swapper/022:38:390
6125992423,1cyclictest34-21ksoftirqd/321:17:243
6125992421,2cyclictest1572-21ssh22:47:283
612599240,24cyclictest0-21swapper/300:27:423
612599240,23cyclictest32221-21diskmemload23:10:013
612599240,23cyclictest13863-21grep21:55:013
612599240,23cyclictest0-21swapper/322:52:323
612599240,23cyclictest0-21swapper/321:23:273
6122992422,1cyclictest313-21systemd-journal00:35:002
6122992420,3cyclictest13575-21apt-get20:33:372
612299240,24cyclictest32221-21diskmemload00:09:472
612299240,0cyclictest7393-21apt-get00:28:372
6114992424,0cyclictest0-21swapper/100:33:351
6114992423,1cyclictest29853-21ssh23:46:161
6114992422,1cyclictest26743-21ssh22:39:291
611499240,24cyclictest32221-21diskmemload21:30:431
611499240,24cyclictest0-21swapper/100:27:481
611499240,1cyclictest7780-21ssh23:56:251
611499240,0cyclictest0-21swapper/121:51:451
611199242,21cyclictest32221-21diskmemload21:50:190
611199240,24cyclictest0-21swapper/023:52:170
611199240,23cyclictest32221-21diskmemload23:01:010
611199240,23cyclictest32221-21diskmemload22:16:060
611199240,23cyclictest32221-21diskmemload22:01:070
611199240,23cyclictest0-21swapper/000:18:400
611199240,1cyclictest32221-21diskmemload22:21:130
6125992323,0cyclictest0-21swapper/323:49:063
6125992321,1cyclictest32221-21diskmemload00:20:203
6125992321,1cyclictest24677-21sh22:05:593
6125992321,1cyclictest0-21swapper/321:26:323
6125992320,2cyclictest20017-21diskmemload23:36:483
612599230,2cyclictest5959-21ssh23:23:363
612599230,23cyclictest0-21swapper/323:19:153
612599230,23cyclictest0-21swapper/322:40:583
612599230,23cyclictest0-21swapper/322:25:163
612599230,22cyclictest8768-21ssh22:22:493
612599230,22cyclictest32221-21diskmemload22:30:353
612599230,22cyclictest26953-1kworker/u9:021:41:453
612599230,22cyclictest17347-21ssh21:58:443
612599230,22cyclictest16072-21sh23:01:053
612599230,22cyclictest0-21swapper/323:51:143
612599230,22cyclictest0-21swapper/319:48:373
612599230,1cyclictest0-21swapper/300:02:163
6122992323,0cyclictest0-21swapper/223:12:092
6122992323,0cyclictest0-21swapper/222:11:112
6122992320,2cyclictest22637-21latency_hist20:50:002
6122992320,0cyclictest0-21swapper/200:00:052
612299230,23cyclictest0-21swapper/223:50:322
612299230,23cyclictest0-21swapper/223:31:382
612299230,23cyclictest0-21swapper/222:28:482
612299230,22cyclictest4964-21latency_hist22:50:012
612299230,22cyclictest24461-21rm23:40:452
612299230,22cyclictest0-21swapper/221:43:462
612299230,1cyclictest0-21swapper/221:26:082
612299230,0cyclictest32221-21diskmemload22:34:052
612299230,0cyclictest32221-21diskmemload22:07:462
6114992323,0cyclictest0-21swapper/123:27:361
6114992323,0cyclictest0-21swapper/122:31:351
6114992323,0cyclictest0-21swapper/121:40:311
6114992323,0cyclictest0-21swapper/100:00:451
6114992322,1cyclictest22397-21ssh23:08:121
6114992321,1cyclictest0-21swapper/121:35:541
6114992320,2cyclictest505-21ssh21:10:541
611499230,2cyclictest3372-21idleruntime-cro20:10:011
611499230,2cyclictest13382-21ssh21:23:411
611499230,23cyclictest32221-21diskmemload22:43:331
611499230,23cyclictest0-21swapper/122:11:111
611499230,22cyclictest8988-21rm22:23:091
611499230,22cyclictest3824-21ssh23:20:081
611499230,22cyclictest32221-21diskmemload21:25:521
611499230,22cyclictest23684-21rm00:12:211
611499230,22cyclictest23684-21rm00:12:211
611499230,22cyclictest20406-21sh22:02:171
611499230,22cyclictest14803-21ssh21:56:241
611499230,22cyclictest0-21swapper/119:38:361
611499230,0cyclictest0-21swapper/123:42:591
611499230,0cyclictest0-21swapper/122:16:241
611499230,0cyclictest0-21swapper/121:17:181
6111992323,0cyclictest32221-21diskmemload00:25:480
6111992323,0cyclictest0-21swapper/023:57:470
6111992323,0cyclictest0-21swapper/023:40:450
6111992323,0cyclictest0-21swapper/023:26:470
6111992323,0cyclictest0-21swapper/021:47:580
6111992323,0cyclictest0-21swapper/021:14:010
6111992323,0cyclictest0-21swapper/000:11:430
6111992323,0cyclictest0-21swapper/000:11:430
6111992323,0cyclictest0-21swapper/000:01:440
6111992322,1cyclictest0-21swapper/022:33:410
6111992321,1cyclictest0-21swapper/022:56:040
6111992321,1cyclictest0-21swapper/022:42:280
611199230,23cyclictest32221-21diskmemload23:17:210
611199230,23cyclictest16010-21ssh00:36:050
611199230,23cyclictest0-21swapper/022:51:040
611199230,23cyclictest0-21swapper/021:27:470
611199230,23cyclictest0-21swapper/021:18:210
611199230,22cyclictest32221-21diskmemload21:36:500
611199230,22cyclictest14306-21ssh21:55:370
611199230,1cyclictest0-21swapper/021:30:150
611199230,1cyclictest0-21swapper/000:31:060
611199230,0cyclictest32221-21diskmemload22:14:220
611199230,0cyclictest0-21swapper/023:47:020
611199230,0cyclictest0-21swapper/022:05:180
6125992222,0cyclictest0-21swapper/323:30:113
6125992222,0cyclictest0-21swapper/300:07:453
6125992220,1cyclictest0-21swapper/322:00:503
6125992220,1cyclictest0-21swapper/300:31:293
612599220,22cyclictest0-21swapper/323:55:073
612599220,22cyclictest0-21swapper/321:11:093
612599220,21cyclictest8248-21turbostat20:20:013
612599220,21cyclictest27419-21ssh00:15:103
612599220,21cyclictest0-21swapper/323:42:123
612599220,21cyclictest0-21swapper/322:55:283
612599220,21cyclictest0-21swapper/320:58:393
612599220,21cyclictest0-21swapper/320:53:363
612599220,21cyclictest0-21swapper/300:10:393
612599220,21cyclictest0-21swapper/300:10:393
612599220,1cyclictest32221-21diskmemload21:48:283
612599220,1cyclictest2625-21ssh22:16:123
6122992222,0cyclictest6105-21cyclictest23:35:162
6122992222,0cyclictest32221-21diskmemload21:39:302
6122992222,0cyclictest0-21swapper/222:50:572
6122992222,0cyclictest0-21swapper/222:41:072
6122992222,0cyclictest0-21swapper/222:21:532
6122992222,0cyclictest0-21swapper/222:01:052
6122992221,1cyclictest32221-21diskmemload00:21:562
6122992220,1cyclictest313-21systemd-journal19:15:002
612299220,2cyclictest29414-21ssh00:18:282
612299220,22cyclictest32303-21sh23:17:112
612299220,22cyclictest0-21swapper/221:57:542
612299220,21cyclictest8928-21sh21:50:062
612299220,21cyclictest27991-21apt-get21:03:382
612299220,21cyclictest26165-21timerandwakeup22:38:472
612299220,21cyclictest25722-21apt19:50:002
612299220,21cyclictest18387-21apt-get20:43:362
612299220,21cyclictest0-21swapper/220:55:002
612299220,21cyclictest0-21swapper/200:35:392
612299220,21cyclictest0-21swapper/200:10:442
612299220,21cyclictest0-21swapper/200:10:442
612299220,1cyclictest20942-21ssh23:05:502
612299220,0cyclictest0-21swapper/223:57:432
612299220,0cyclictest0-21swapper/223:01:102
612299220,0cyclictest0-21swapper/222:58:172
612299220,0cyclictest0-21swapper/221:45:552
6114992222,0cyclictest32221-21diskmemload21:46:431
6114992222,0cyclictest0-21swapper/123:53:431
6114992222,0cyclictest0-21swapper/122:25:261
6114992222,0cyclictest0-21swapper/122:06:521
6114992220,1cyclictest1-21systemd19:20:001
6114992219,2cyclictest25747-21cut19:50:001
611499220,22cyclictest0-21swapper/100:24:041
611499220,21cyclictest31529-21apt-get20:03:371
611499220,21cyclictest23330-21latency_hist19:45:011
611499220,21cyclictest16021-21sh23:01:001
611499220,21cyclictest11481-21apt-get19:23:361
611499220,21cyclictest0-21swapper/100:15:281
611499220,1cyclictest17526-21apt-get00:38:381
6111992222,0cyclictest0-21swapper/020:53:360
6111992222,0cyclictest0-21swapper/000:06:250
611199220,21cyclictest27449-21latency_hist21:00:010
611199220,21cyclictest0-21swapper/023:35:050
611199220,21cyclictest0-21swapper/023:06:430
611199220,21cyclictest0-21swapper/000:20:090
611199220,1cyclictest32221-21diskmemload21:41:070
611199220,1cyclictest30384-21apt-get21:08:390
611199220,1cyclictest27781-21rm23:13:050
611199220,1cyclictest0-21swapper/023:32:080
6125992120,1cyclictest331rcuc/320:25:013
6125992120,0cyclictest13873-21apt-get19:28:363
6125992119,1cyclictest10939-21sendmail-msp19:20:003
6125992119,1cyclictest0-21swapper/300:37:063
612599210,21cyclictest0-21swapper/321:35:043
612599210,20cyclictest0-21swapper/319:55:023
612599210,1cyclictest0-21swapper/322:10:073
612599210,0cyclictest32221-21diskmemload23:26:003
6122992121,0cyclictest2328-21ssh22:15:462
6122992120,1cyclictest20966-21lockfile-remove19:40:012
612299210,21cyclictest32221-21diskmemload21:31:402
612299210,20cyclictest0-21swapper/221:08:392
612299210,0cyclictest0-21swapper/223:45:342
6114992121,0cyclictest0-21swapper/123:35:331
6114992121,0cyclictest0-21swapper/122:55:061
6114992121,0cyclictest0-21swapper/100:05:171
6114992120,1cyclictest23178-21apt-get20:53:361
6114992119,1cyclictest13042-21perf20:30:011
6111992119,1cyclictest22656-21munin-run20:50:000
611199210,21cyclictest0-21swapper/019:30:020
611199210,21cyclictest0-21swapper/019:23:360
611199210,1cyclictest11173-21apt-get20:28:360
611199210,0cyclictest8781-21apt-get20:23:360
6125992020,0cyclictest15746-21apt-get19:30:023
612599200,0cyclictest0-21swapper/320:03:373
6122992020,0cyclictest0-21swapper/220:23:362
6114992020,0cyclictest25574-21apt-get20:58:391
6114992020,0cyclictest0-21swapper/121:10:011
6114992020,0cyclictest0-21swapper/119:58:371
6114992020,0cyclictest0-21swapper/119:13:361
6111992020,0cyclictest3905-21apt-get20:13:360
6111992020,0cyclictest0-21swapper/020:43:360
6111992020,0cyclictest0-21swapper/019:43:370
6111992019,1cyclictest17867-21idleruntime-cro20:40:000
611199200,0cyclictest0-21swapper/020:18:380
6122991919,0cyclictest0-21swapper/220:28:362
6114991917,1cyclictest0-21swapper/120:50:001
6125991818,0cyclictest0-21swapper/319:38:363
6125991815,2cyclictest29863-21munin-run21:05:013
6125991815,2cyclictest15424-21awk20:35:013
6122991818,0cyclictest6312-21apt-get20:18:382
6122991818,0cyclictest0-21swapper/220:58:392
6122991818,0cyclictest0-21swapper/219:28:362
6114991818,0cyclictest16674-21apt-get19:33:371
6114991818,0cyclictest13873-21apt-get19:28:361
6114991818,0cyclictest0-21swapper/120:33:371
6111991817,1cyclictest28185-21apt-get19:55:020
6111991816,1cyclictest0-21swapper/019:50:000
611199181,16cyclictest3347-21cron20:10:000
6125991717,0cyclictest0-21swapper/320:43:363
6125991717,0cyclictest0-21swapper/320:08:373
6122991717,0cyclictest0-21swapper/220:03:372
6122991717,0cyclictest0-21swapper/219:18:372
612299170,0cyclictest0-21swapper/220:08:362
6114991717,0cyclictest0-21swapper/120:38:361
6125991615,1cyclictest6667-21apt-get19:13:363
612599160,1cyclictest0-21swapper/319:43:373
6122991615,1cyclictest17866-21sendmail-msp20:40:002
6111991616,0cyclictest0-21swapper/019:28:360
6122991515,0cyclictest0-21swapper/220:13:362
6122991515,0cyclictest0-21swapper/219:58:372
6122991515,0cyclictest0-21swapper/219:33:372
612299150,1cyclictest27321-21ntp_states19:53:412
6111991515,0cyclictest0-21swapper/019:53:350
6111991515,0cyclictest0-21swapper/019:18:370
6125991414,0cyclictest0-21swapper/320:10:053
612599140,14cyclictest0-21swapper/321:05:453
612599140,14cyclictest0-21swapper/320:35:023
612599140,14cyclictest0-21swapper/319:50:233
612599140,0cyclictest0-21swapper/320:45:133
612599140,0cyclictest0-21swapper/320:25:433
612299140,14cyclictest0-21swapper/219:40:092
612299140,14cyclictest0-21swapper/219:20:232
6114991414,0cyclictest0-21swapper/121:00:491
6114991414,0cyclictest0-21swapper/120:15:291
611499140,14cyclictest0-21swapper/119:50:191
611499140,0cyclictest0-21swapper/120:10:561
6111991414,0cyclictest0-21swapper/020:00:440
611199140,14cyclictest0-21swapper/021:00:320
611199140,14cyclictest0-21swapper/019:36:120
611199140,14cyclictest0-21swapper/019:12:200
611199140,0cyclictest0-21swapper/020:30:360
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional