You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-18 - 10:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon May 18, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
537199280,27cyclictest0-21swapper/123:52:251
537199270,26cyclictest31819-21diskmemload00:01:371
537799260,25cyclictest0-21swapper/322:23:163
537699260,25cyclictest31819-21diskmemload21:57:532
5371992623,3cyclictest0-21swapper/121:47:431
537199260,25cyclictest29615-21ssh22:05:391
537199260,25cyclictest2290-21rm22:38:471
537199260,25cyclictest0-21swapper/121:15:201
537799252,22cyclictest24929-21sh22:30:253
537799250,24cyclictest31819-21diskmemload22:29:563
537799250,24cyclictest31819-21diskmemload22:07:193
537799250,24cyclictest0-21swapper/321:17:193
537699250,24cyclictest8785-21perf00:40:002
537699250,24cyclictest31819-21diskmemload23:39:332
537699250,24cyclictest23682-21ssh00:24:202
537699250,24cyclictest0-21swapper/223:13:142
537699250,24cyclictest0-21swapper/222:37:462
537699250,24cyclictest0-21swapper/222:29:242
537699250,0cyclictest0-21swapper/220:02:252
5371992523,2cyclictest0-21swapper/100:28:191
5371992522,2cyclictest0-21swapper/100:07:451
5371992521,3cyclictest20338-21ssh00:22:081
537199250,25cyclictest0-21swapper/100:35:271
537199250,24cyclictest31819-21diskmemload22:41:471
537199250,24cyclictest15769-21ssh23:48:291
537199250,24cyclictest0-21swapper/121:58:511
537199250,24cyclictest0-21swapper/121:23:111
537199250,1cyclictest15334-21sh22:51:021
537199250,0cyclictest31819-21diskmemload22:32:181
536499250,24cyclictest2135-21cron20:10:010
537799241,23cyclictest7410-21apt-get20:20:023
537799240,23cyclictest5853-21ssh23:11:363
537799240,23cyclictest31819-21diskmemload22:03:103
537799240,23cyclictest31819-21diskmemload00:03:163
537799240,23cyclictest0-21swapper/323:22:013
537799240,23cyclictest0-21swapper/321:26:493
537799240,0cyclictest0-21swapper/321:12:133
5376992420,4cyclictest0-21swapper/221:41:442
537699240,24cyclictest4974-1kworker/u9:122:16:422
537699240,23cyclictest12463-21ssh00:14:172
537699240,23cyclictest0-21swapper/200:17:062
537699240,1cyclictest0-21swapper/221:33:212
5371992421,3cyclictest0-21swapper/122:12:301
537199240,24cyclictest0-21swapper/123:16:581
537199240,24cyclictest0-21swapper/122:21:501
537199240,23cyclictest17395-21ssh21:26:201
537199240,20cyclictest19970-21kworker/u8:223:32:561
537199240,1cyclictest11987-21ssh22:18:141
537199240,0cyclictest25298-21interrupts23:57:291
537199240,0cyclictest0-21swapper/123:43:321
537199240,0cyclictest0-21swapper/123:00:351
537199240,0cyclictest0-21swapper/122:49:311
537199240,0cyclictest0-21swapper/122:03:451
537199240,0cyclictest0-21swapper/121:50:591
537199240,0cyclictest0-21swapper/100:18:381
5364992424,0cyclictest0-21swapper/000:26:150
5364992423,1cyclictest31819-21diskmemload22:32:170
536499240,24cyclictest0-21swapper/022:00:080
536499240,24cyclictest0-21swapper/021:37:430
536499240,23cyclictest62550irq/127-eno121:58:270
536499240,1cyclictest31819-21diskmemload23:26:050
536499240,0cyclictest0-21swapper/023:05:110
536499240,0cyclictest0-21swapper/023:01:380
5377992323,0cyclictest31819-21diskmemload23:16:543
5377992323,0cyclictest31819-21diskmemload21:33:243
5377992323,0cyclictest0-21swapper/323:54:343
5377992323,0cyclictest0-21swapper/323:48:293
5377992323,0cyclictest0-21swapper/323:41:353
5377992323,0cyclictest0-21swapper/322:45:153
5377992323,0cyclictest0-21swapper/321:55:213
5377992323,0cyclictest0-21swapper/321:52:243
5377992321,1cyclictest31819-21diskmemload23:27:323
5377992321,1cyclictest0-21swapper/322:50:503
5377992321,1cyclictest0-21swapper/321:35:333
537799231,1cyclictest26708-21sh23:58:053
537799230,23cyclictest0-21swapper/323:00:013
537799230,22cyclictest4224-21ssh00:36:193
537799230,22cyclictest31819-21diskmemload23:37:343
537799230,22cyclictest31819-21diskmemload22:55:463
537799230,22cyclictest31819-21diskmemload21:45:083
537799230,22cyclictest27008-21grep21:00:013
537799230,22cyclictest1405-21sh21:40:553
537799230,1cyclictest0-21swapper/322:16:473
537799230,0cyclictest31819-21diskmemload22:13:143
537799230,0cyclictest17368-21ssh00:18:173
537799230,0cyclictest0-21swapper/300:09:003
5376992323,0cyclictest0-21swapper/223:05:302
5376992322,0cyclictest31819-21diskmemload21:16:082
5376992322,0cyclictest0-21swapper/219:52:242
5376992320,0cyclictest0-21swapper/222:20:292
537699231,22cyclictest418-21ssh21:11:162
537699230,23cyclictest0-21swapper/223:28:362
537699230,23cyclictest0-21swapper/222:46:322
537699230,23cyclictest0-21swapper/200:03:572
537699230,22cyclictest7423-21cron20:20:002
537699230,22cyclictest31819-21diskmemload23:59:352
537699230,22cyclictest28250-21ssh21:35:552
537699230,22cyclictest27803-21apt-get19:57:232
537699230,22cyclictest19087-21ssh23:52:252
537699230,22cyclictest0-21swapper/223:46:402
537699230,22cyclictest0-21swapper/223:22:252
537699230,0cyclictest0-21swapper/222:53:252
537699230,0cyclictest0-21swapper/222:40:242
537699230,0cyclictest0-21swapper/221:48:292
5371992323,0cyclictest0-21swapper/123:12:411
5371992323,0cyclictest0-21swapper/121:32:161
5371992320,3cyclictest0-21swapper/122:58:451
537199230,23cyclictest0-21swapper/123:26:341
537199230,23cyclictest0-21swapper/123:08:371
537199230,23cyclictest0-21swapper/122:25:181
537199230,23cyclictest0-21swapper/100:12:201
537199230,20cyclictest0-21swapper/100:32:351
537199230,0cyclictest0-21swapper/123:35:361
537199230,0cyclictest0-21swapper/123:21:561
537199230,0cyclictest0-21swapper/120:37:251
5364992323,0cyclictest0-21swapper/023:59:110
5364992323,0cyclictest0-21swapper/023:10:310
5364992323,0cyclictest0-21swapper/022:55:170
5364992323,0cyclictest0-21swapper/000:08:240
5364992322,1cyclictest0-21swapper/022:54:090
5364992322,1cyclictest0-21swapper/021:25:280
5364992321,1cyclictest0-21swapper/021:42:260
536499230,23cyclictest0-21swapper/023:30:490
536499230,23cyclictest0-21swapper/021:53:120
536499230,23cyclictest0-21swapper/021:47:030
536499230,23cyclictest0-21swapper/000:21:120
536499230,23cyclictest0-21swapper/000:00:290
536499230,22cyclictest62550irq/127-eno122:44:290
536499230,22cyclictest31819-21diskmemload23:16:030
536499230,22cyclictest313-21systemd-journal20:40:000
536499230,22cyclictest22215-21latency_hist20:50:010
536499230,22cyclictest10970-21awk00:12:340
536499230,22cyclictest0-21swapper/022:15:300
536499230,22cyclictest0-21swapper/000:32:250
536499230,1cyclictest31819-21diskmemload21:10:030
536499230,1cyclictest0-21swapper/022:05:200
536499230,1cyclictest0-21swapper/021:34:510
536499230,1cyclictest0-21swapper/000:40:010
536499230,1cyclictest0-21swapper/000:17:570
536499230,0cyclictest0-21swapper/022:48:340
536499230,0cyclictest0-21swapper/022:21:250
5377992222,0cyclictest0-21swapper/322:40:033
5377992222,0cyclictest0-21swapper/300:30:333
5377992221,1cyclictest0-21swapper/323:07:163
5377992221,0cyclictest0-21swapper/320:47:243
537799220,21cyclictest11300-21sh21:20:403
537799220,21cyclictest0-21swapper/320:52:233
537799220,21cyclictest0-21swapper/319:30:003
537799220,21cyclictest0-21swapper/300:26:103
537799220,21cyclictest0-21swapper/300:22:243
537799220,1cyclictest10092-21/usr/sbin/munin00:12:293
537799220,1cyclictest0-21swapper/323:30:173
537799220,1cyclictest0-21swapper/322:36:323
5376992222,0cyclictest0-21swapper/223:30:082
5376992222,0cyclictest0-21swapper/221:50:252
5376992222,0cyclictest0-21swapper/221:22:382
5376992222,0cyclictest0-21swapper/200:28:192
5376992220,1cyclictest26340-21apt-get22:32:222
537699220,21cyclictest9320-21sh23:42:352
537699220,21cyclictest31819-21diskmemload22:55:062
537699220,21cyclictest31819-21diskmemload22:10:472
537699220,21cyclictest31819-21diskmemload22:01:492
537699220,21cyclictest30345-21ssh00:30:192
537699220,21cyclictest15004-21awk20:35:002
537699220,21cyclictest1347-21apt-get00:05:022
537699220,21cyclictest10302-21ssh23:15:012
537699220,1cyclictest8603-21apt-get19:17:252
537699220,1cyclictest0-21swapper/223:01:492
537699220,0cyclictest0-21swapper/221:25:362
537199220,21cyclictest20622-21apt-get19:42:251
537199220,21cyclictest15021-21wc20:35:001
537199220,1cyclictest28682-21ssh21:36:391
537199220,1cyclictest1185-21ssh21:12:121
5364992222,0cyclictest31819-21diskmemload23:45:140
5364992222,0cyclictest0-21swapper/022:25:320
5364992220,1cyclictest15032-21munin-run20:35:000
5364992220,1cyclictest0-21swapper/022:10:280
5364992219,2cyclictest6599-21sh23:40:550
536499220,22cyclictest0-21swapper/022:35:350
536499220,21cyclictest62550irq/127-eno123:35:400
536499220,21cyclictest27376-21munin-run19:55:010
536499220,21cyclictest18206-21apt-get19:37:240
536499220,21cyclictest10785-21ssh21:20:010
536499220,21cyclictest0-21swapper/019:57:240
536499220,21cyclictest0-21swapper/019:47:250
536499220,1cyclictest6661-21sh21:17:020
536499220,1cyclictest31819-21diskmemload23:20:260
536499220,1cyclictest0-21swapper/023:51:250
537799210,20cyclictest0-21swapper/319:47:253
537799210,1cyclictest29427-21latency_hist21:05:013
537799210,1cyclictest0-21swapper/319:57:243
5376992119,2cyclictest4959-21latency_hist20:15:012
537699210,20cyclictest31819-21diskmemload22:05:152
537699210,20cyclictest27475-21apt-get21:02:252
537699210,20cyclictest0-21swapper/221:07:252
537699210,20cyclictest0-21swapper/220:20:022
537699210,20cyclictest0-21swapper/219:42:252
537699210,0cyclictest0-21swapper/219:32:242
537199210,0cyclictest0-21swapper/120:17:261
537199210,0cyclictest0-21swapper/120:12:251
5364992121,0cyclictest0-21swapper/019:27:240
5364992119,1cyclictest0-21swapper/021:05:010
536499210,20cyclictest10665-21apt-get20:27:250
536499210,0cyclictest0-21swapper/021:07:250
536499210,0cyclictest0-21swapper/019:17:250
5377992020,0cyclictest0-21swapper/319:15:023
5377992019,1cyclictest20622-21apt-get19:42:253
537799200,20cyclictest0-21swapper/321:10:003
537799200,20cyclictest0-21swapper/319:32:243
5376992020,0cyclictest0-21swapper/220:37:252
5376992020,0cyclictest0-21swapper/219:22:242
537699200,0cyclictest0-21swapper/220:47:252
537699200,0cyclictest0-21swapper/219:47:252
5371992020,0cyclictest32617-21apt-get20:07:231
5371992020,0cyclictest17875-21apt-get20:42:251
5371992020,0cyclictest0-21swapper/121:02:261
5371992019,1cyclictest24618-21perf20:55:011
537199200,20cyclictest0-21swapper/119:15:021
5364992020,0cyclictest5813-21apt-get19:12:260
5364992020,0cyclictest5407-21apt-get20:17:260
5364992020,0cyclictest0-21swapper/020:42:250
5364992020,0cyclictest0-21swapper/019:22:240
536499200,0cyclictest0-21swapper/020:22:240
5377991917,1cyclictest7410-21apt-get20:20:003
5377991917,1cyclictest32165-21cron20:05:013
5376991917,1cyclictest0-21swapper/219:40:002
5371991919,0cyclictest20267-21apt-get20:47:241
5371991919,0cyclictest0-21swapper/121:07:251
5371991919,0cyclictest0-21swapper/120:27:251
5371991919,0cyclictest0-21swapper/119:47:251
5371991919,0cyclictest0-21swapper/119:37:251
5371991919,0cyclictest0-21swapper/119:22:231
5364991919,0cyclictest0-21swapper/020:52:230
5377991818,0cyclictest0-21swapper/320:42:253
5377991818,0cyclictest0-21swapper/320:10:023
5376991818,0cyclictest0-21swapper/220:42:252
5376991816,1cyclictest313-21systemd-journal20:10:012
5371991818,0cyclictest25057-21apt-get20:57:251
5371991815,3cyclictest0-21swapper/119:12:261
5377991716,1cyclictest0-21swapper/320:37:243
5371991717,0cyclictest8269-21apt-get20:22:241
5364991717,0cyclictest0-21swapper/019:42:250
5377991616,0cyclictest0-21swapper/319:12:263
5377991614,1cyclictest0-21swapper/320:32:343
5376991616,0cyclictest0-21swapper/220:57:252
5371991616,0cyclictest0-21swapper/119:27:231
5371991614,2cyclictest0-21swapper/119:55:041
537199160,16cyclictest0-21swapper/119:33:301
5376991515,0cyclictest0-21swapper/219:12:262
5376991513,1cyclictest0-21swapper/220:27:302
537699150,15cyclictest22920-21cstates20:52:242
537699150,14cyclictest13654-21expr19:27:242
5371991514,1cyclictest0-21swapper/120:02:501
536499157,8cyclictest17763-21sh19:35:000
5364991513,1cyclictest0-21swapper/020:57:270
537799140,14cyclictest0-21swapper/320:05:103
537799140,14cyclictest0-21swapper/319:50:473
537799140,14cyclictest0-21swapper/319:21:513
537799140,0cyclictest0-21swapper/320:25:143
537799140,0cyclictest0-21swapper/319:35:193
536499140,0cyclictest0-21swapper/020:01:430
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional