You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 12:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Wed May 13, 2026 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3023699287,1cyclictest34-21ksoftirqd/322:29:483
30222992625,1cyclictest62550irq/127-eno123:40:390
3023699250,24cyclictest3697-21ps22:17:063
3022599250,24cyclictest23861-21diskmemload00:21:071
3022299250,24cyclictest23861-21diskmemload23:16:440
3022299250,1cyclictest31558-21ssh23:10:130
30236992421,2cyclictest23861-21diskmemload22:37:303
3023699240,23cyclictest911-21diskstats23:12:013
3023699240,23cyclictest0-21swapper/323:27:473
3023699240,23cyclictest0-21swapper/322:48:493
3023699240,23cyclictest0-21swapper/300:22:013
3023699240,1cyclictest26337-21ssh23:33:203
3023699240,1cyclictest0-21swapper/322:21:273
30230992423,1cyclictest0-21swapper/223:12:092
3023099240,24cyclictest0-21swapper/221:36:422
3023099240,23cyclictest23861-21diskmemload00:27:112
3023099240,23cyclictest0-21swapper/223:58:322
3023099240,23cyclictest0-21swapper/223:27:072
3023099240,23cyclictest0-21swapper/221:30:462
3023099240,1cyclictest0-21swapper/223:02:162
30225992424,0cyclictest0-21swapper/123:46:541
30225992424,0cyclictest0-21swapper/122:55:511
30225992424,0cyclictest0-21swapper/122:07:321
30225992423,1cyclictest0-21swapper/123:42:131
30225992422,1cyclictest0-21swapper/121:52:021
3022599240,24cyclictest23861-21diskmemload23:11:481
3022599240,23cyclictest7025-21diskmemload22:20:361
3022599240,23cyclictest23861-21diskmemload21:31:291
3022599240,23cyclictest22233-21turbostat20:00:001
3022599240,23cyclictest14126-21ssh21:57:281
3022599240,0cyclictest23861-21diskmemload00:34:211
3022599240,0cyclictest0-21swapper/122:12:281
30222992421,2cyclictest0-21swapper/000:25:300
3022299240,24cyclictest0-21swapper/021:50:570
3022299240,23cyclictest26840-21rm22:09:200
3022299240,0cyclictest0-21swapper/022:48:430
30236992323,0cyclictest0-21swapper/323:37:223
30236992323,0cyclictest0-21swapper/322:58:593
30236992323,0cyclictest0-21swapper/322:33:133
30236992323,0cyclictest0-21swapper/322:33:133
30236992323,0cyclictest0-21swapper/322:13:453
30236992323,0cyclictest0-21swapper/321:39:513
30236992322,1cyclictest8459-21kworker/3:122:42:193
30236992322,1cyclictest34-21ksoftirqd/323:23:413
30236992322,1cyclictest23861-21diskmemload23:57:413
30236992322,1cyclictest0-21swapper/300:34:273
30236992321,1cyclictest34-21ksoftirqd/322:53:123
30236992321,1cyclictest0-21swapper/321:50:073
30236992320,2cyclictest30237-21sh21:15:353
3023699230,23cyclictest0-21swapper/321:20:063
3023699230,22cyclictest926-21cstates00:37:003
3023699230,22cyclictest26342-21rm23:05:533
3023699230,22cyclictest20437-21ssh22:03:153
3023699230,22cyclictest14806-21ssh23:52:033
3023699230,22cyclictest0-21swapper/323:43:383
3023699230,0cyclictest23861-21diskmemload23:45:173
3023699230,0cyclictest23861-21diskmemload21:13:023
3023699230,0cyclictest0-21swapper/321:30:133
3023699230,0cyclictest0-21swapper/300:01:233
30230992323,0cyclictest0-21swapper/223:36:532
30230992323,0cyclictest0-21swapper/223:31:592
30230992323,0cyclictest0-21swapper/222:52:232
30230992323,0cyclictest0-21swapper/221:17:262
30230992323,0cyclictest0-21swapper/200:38:162
30230992321,1cyclictest29123-21apt-get22:11:592
30230992321,1cyclictest2502-21ssh00:09:502
30230992321,1cyclictest2502-21ssh00:09:502
30230992321,1cyclictest0-21swapper/223:46:062
30230992321,1cyclictest0-21swapper/221:55:512
30230992321,1cyclictest0-21swapper/200:33:452
3023099230,23cyclictest0-21swapper/222:30:202
3023099230,23cyclictest0-21swapper/222:30:202
3023099230,23cyclictest0-21swapper/200:10:322
3023099230,22cyclictest6114-21ssh21:51:422
3023099230,22cyclictest5179-21sh23:43:112
3023099230,22cyclictest23861-21diskmemload22:24:492
3023099230,22cyclictest23861-21diskmemload22:06:082
3023099230,22cyclictest23861-21diskmemload00:02:322
3023099230,22cyclictest0-21swapper/222:15:392
3023099230,22cyclictest0-21swapper/220:40:002
3023099230,0cyclictest0-21swapper/223:51:452
3023099230,0cyclictest0-21swapper/222:55:412
3023099230,0cyclictest0-21swapper/222:40:242
3023099230,0cyclictest0-21swapper/222:37:302
3023099230,0cyclictest0-21swapper/222:28:482
30225992323,0cyclictest23861-21diskmemload23:36:421
30225992323,0cyclictest23861-21diskmemload21:10:091
30225992323,0cyclictest0-21swapper/122:15:331
30225992323,0cyclictest0-21swapper/121:25:571
30225992322,1cyclictest9476-21ssh22:51:061
3022599230,23cyclictest23861-21diskmemload00:01:271
3022599230,23cyclictest0-21swapper/123:50:031
3022599230,23cyclictest0-21swapper/122:01:161
3022599230,23cyclictest0-21swapper/100:10:041
3022599230,22cyclictest23861-21diskmemload23:20:171
3022599230,22cyclictest23861-21diskmemload23:00:381
3022599230,0cyclictest23861-21diskmemload22:35:141
3022599230,0cyclictest0-21swapper/123:18:151
3022599230,0cyclictest0-21swapper/122:46:301
3022599230,0cyclictest0-21swapper/100:35:161
30222992323,0cyclictest62550irq/127-eno100:33:360
30222992323,0cyclictest23861-21diskmemload22:30:270
30222992323,0cyclictest23861-21diskmemload22:30:270
30222992323,0cyclictest23861-21diskmemload22:01:460
30222992323,0cyclictest23861-21diskmemload21:31:230
30222992323,0cyclictest23861-21diskmemload21:11:520
30222992323,0cyclictest0-21swapper/023:39:070
30222992323,0cyclictest0-21swapper/022:18:420
30222992323,0cyclictest0-21swapper/000:15:180
30222992321,1cyclictest23861-21diskmemload21:56:350
30222992321,1cyclictest0-21swapper/019:55:000
30222992310,13cyclictest0-21swapper/020:22:000
3022299230,23cyclictest23861-21diskmemload23:34:550
3022299230,23cyclictest23861-21diskmemload22:50:510
3022299230,23cyclictest0-21swapper/023:57:040
3022299230,22cyclictest62550irq/127-eno123:47:320
3022299230,22cyclictest4033-21rm00:11:340
3022299230,22cyclictest31463-21cat22:42:030
3022299230,22cyclictest29897-21ssh23:08:190
3022299230,22cyclictest20164-21rm23:00:110
3022299230,22cyclictest11601-21ssh23:21:060
3022299230,22cyclictest0-21swapper/021:15:040
3022299230,0cyclictest23861-21diskmemload22:28:520
3022299230,0cyclictest0-21swapper/022:36:220
3022299230,0cyclictest0-21swapper/021:36:550
3022299230,0cyclictest0-21swapper/021:28:300
3022299230,0cyclictest0-21swapper/000:00:310
30236992221,1cyclictest25040-21apt-get20:07:003
30236992221,1cyclictest25040-21apt-get20:07:003
30236992221,1cyclictest23861-21diskmemload21:40:023
30236992221,1cyclictest0-21swapper/322:05:053
30236992220,1cyclictest4283-21perf20:30:003
30236992220,1cyclictest0-21swapper/323:15:033
3023699220,21cyclictest31836-21latency_hist20:20:013
3023699220,21cyclictest0-21swapper/321:46:313
3023699220,21cyclictest0-21swapper/300:15:173
3023699220,21cyclictest0-21swapper/300:10:113
3023699220,1cyclictest0-21swapper/321:57:023
3023699220,1cyclictest0-21swapper/321:25:203
3023699220,1cyclictest0-21swapper/300:26:383
3023699220,0cyclictest0-21swapper/323:00:123
3023699220,0cyclictest0-21swapper/300:07:203
3023699220,0cyclictest0-21swapper/300:07:203
30230992222,0cyclictest0-21swapper/221:27:202
30230992221,1cyclictest0-21swapper/200:16:572
30230992220,1cyclictest0-21swapper/223:17:022
3023099220,21cyclictest4631-21apt-get21:22:002
3023099220,21cyclictest0-21swapper/223:21:182
3023099220,21cyclictest0-21swapper/223:06:032
3023099220,21cyclictest0-21swapper/222:00:292
3023099220,21cyclictest0-21swapper/221:45:142
3023099220,21cyclictest0-21swapper/221:10:352
3023099220,1cyclictest2817-21dpkg22:45:012
3023099220,1cyclictest23889-21perf21:10:012
3023099220,1cyclictest0-21swapper/200:20:022
3023099220,0cyclictest0-21swapper/220:07:002
3023099220,0cyclictest0-21swapper/220:07:002
30225992222,0cyclictest0-21swapper/123:30:251
30225992222,0cyclictest0-21swapper/121:17:471
30225992222,0cyclictest0-21swapper/100:27:351
30225992222,0cyclictest0-21swapper/100:07:181
30225992222,0cyclictest0-21swapper/100:07:181
30225992220,1cyclictest32562-21cron19:15:001
3022599220,2cyclictest26620-21ssh23:06:141
3022599220,2cyclictest17694-21rm23:26:441
3022599220,22cyclictest26456-21rm21:40:391
3022599220,22cyclictest211rcuc/119:36:561
3022599220,21cyclictest9901-21apt-get20:41:591
3022599220,21cyclictest0-21swapper/122:40:401
3022599220,21cyclictest0-21swapper/121:45:331
3022599220,21cyclictest0-21swapper/120:01:581
3022599220,1cyclictest606-21dbus-daemon19:35:011
3022599220,1cyclictest0-21swapper/119:45:001
3022599220,0cyclictest0-21swapper/122:31:311
3022599220,0cyclictest0-21swapper/122:31:311
3022599220,0cyclictest0-21swapper/121:21:141
30222992222,0cyclictest0-21swapper/021:46:130
30222992222,0cyclictest0-21swapper/000:35:180
3022299220,21cyclictest26003-21ssh21:40:050
3022299220,21cyclictest0-21swapper/019:36:560
3022299220,1cyclictest62550irq/127-eno121:10:010
3022299220,1cyclictest23861-21diskmemload22:55:070
3022299220,1cyclictest23861-21diskmemload22:20:460
3022299220,1cyclictest1502-21ssh00:08:290
3022299220,1cyclictest1502-21ssh00:08:290
3022299220,0cyclictest0-21swapper/022:10:070
30236992121,0cyclictest0-21swapper/320:01:583
30236992120,1cyclictest34-21ksoftirqd/319:41:583
3023699210,20cyclictest0-21swapper/320:51:583
3023699210,1cyclictest0-21swapper/319:46:583
30230992120,1cyclictest2599-21awk19:20:012
30230992120,1cyclictest0-21swapper/219:41:582
30230992120,0cyclictest0-21swapper/219:26:592
3023099210,20cyclictest30586-21apt-get19:11:592
3023099210,1cyclictest1841-21perf20:25:012
3023099210,1cyclictest0-21swapper/220:57:002
30225992121,0cyclictest27433-21apt-get20:11:591
30225992121,0cyclictest0-21swapper/122:25:221
30225992121,0cyclictest0-21swapper/120:16:581
30225992121,0cyclictest0-21swapper/100:15:171
30225992119,1cyclictest19089-21perf21:00:011
3022599210,21cyclictest20463-21ssh21:35:191
3022599210,21cyclictest18151-21ssh23:55:031
3022599210,0cyclictest19500-21apt-get21:01:581
30222992121,0cyclictest5441-21apt-get19:26:580
30222992121,0cyclictest0-21swapper/021:20:030
30222992121,0cyclictest0-21swapper/020:40:020
3022299210,21cyclictest0-21swapper/023:25:130
3022299210,20cyclictest0-21swapper/020:15:010
3022299210,0cyclictest23861-21diskmemload00:20:030
3022299210,0cyclictest0-21swapper/020:07:000
3022299210,0cyclictest0-21swapper/020:07:000
30236992020,0cyclictest0-21swapper/321:01:583
30236992020,0cyclictest0-21swapper/319:11:593
30236992018,1cyclictest0-21swapper/321:10:013
3023699200,20cyclictest0-21swapper/320:57:003
3023699200,20cyclictest0-21swapper/319:31:583
3023699200,0cyclictest0-21swapper/320:46:593
3023699200,0cyclictest0-21swapper/320:11:583
30230992020,0cyclictest3048-21apt-get19:22:002
30230992020,0cyclictest17832-21apt-get19:51:582
30230992019,1cyclictest20227-21apt-get19:56:582
30230992018,1cyclictest0-21swapper/220:05:012
30225992020,0cyclictest0-21swapper/120:46:591
30225992020,0cyclictest0-21swapper/120:26:571
30225992020,0cyclictest0-21swapper/119:51:581
30225992020,0cyclictest0-21swapper/119:16:591
3022599200,0cyclictest25040-21apt-get20:07:001
3022599200,0cyclictest25040-21apt-get20:07:001
30222992020,0cyclictest0-21swapper/020:37:000
3022299200,20cyclictest0-21swapper/020:51:580
3022299200,0cyclictest0-21swapper/019:31:590
30236991919,0cyclictest0-21swapper/319:56:573
30236991919,0cyclictest0-21swapper/319:21:593
30230991919,0cyclictest2239-21apt-get20:26:572
30230991919,0cyclictest0-21swapper/220:31:572
30230991919,0cyclictest0-21swapper/219:46:592
3023099190,0cyclictest0-21swapper/221:01:582
30225991919,0cyclictest15044-21apt-get19:46:591
30225991919,0cyclictest14689-21apt-get20:51:581
30222991919,0cyclictest0-21swapper/021:01:580
3022299190,1cyclictest0-21swapper/019:42:000
30230991818,0cyclictest0-21swapper/220:40:022
3023099180,16cyclictest313-21systemd-journal20:54:562
30225991818,0cyclictest0-21swapper/121:07:021
30222991818,0cyclictest0-21swapper/020:31:570
30236991717,0cyclictest0-21swapper/320:37:003
30236991717,0cyclictest0-21swapper/320:21:593
30236991716,1cyclictest0-21swapper/320:41:583
30230991717,0cyclictest0-21swapper/220:16:592
30225991717,0cyclictest0-21swapper/120:37:001
30222991717,0cyclictest521-21apt-get19:16:580
30222991717,0cyclictest30586-21apt-get19:11:590
30222991717,0cyclictest17086-21apt-get20:57:000
30222991715,1cyclictest14255-21cron20:50:000
30236991616,0cyclictest0-21swapper/319:51:583
30236991616,0cyclictest0-21swapper/319:36:563
30230991616,0cyclictest0-21swapper/220:11:592
30236991515,0cyclictest0-21swapper/319:16:583
30230991515,0cyclictest0-21swapper/219:31:582
30230991514,1cyclictest12229-21perf19:40:002
30225991515,0cyclictest0-21swapper/119:26:591
30225991514,1cyclictest6697-21awk20:35:001
3022599150,5cyclictest0-21swapper/119:22:001
30222991515,0cyclictest0-21swapper/020:16:590
30222991515,0cyclictest0-21swapper/020:01:580
3023699140,0cyclictest0-21swapper/320:31:163
3023699140,0cyclictest0-21swapper/319:25:163
3023099140,14cyclictest0-21swapper/220:45:322
3022599140,14cyclictest0-21swapper/120:20:251
30222991414,0cyclictest0-21swapper/019:45:290
3022299140,0cyclictest0-21swapper/020:26:050
3022299140,0cyclictest0-21swapper/019:55:170
3022299140,0cyclictest0-21swapper/019:20:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional