You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-15 - 05:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Wed Apr 15, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2201199260,25cyclictest18488-21ssh23:18:582
2200499264,22cyclictest13-21ksoftirqd/023:14:160
2201599250,24cyclictest16029-21diskmemload00:29:043
2201599250,24cyclictest0-21swapper/321:10:103
22011992523,1cyclictest29119-21apt-get23:29:382
2201199250,24cyclictest16029-21diskmemload22:32:562
2201199250,24cyclictest10190-21ssh23:40:562
22004992523,1cyclictest17816-21ssh21:44:300
2201599240,23cyclictest7588-21ssh22:04:443
2201599240,23cyclictest20767-21rm23:19:553
2201599240,23cyclictest0-21swapper/321:39:393
2201599240,1cyclictest0-21swapper/322:44:443
2201599240,0cyclictest16029-21diskmemload21:43:243
22011992423,0cyclictest1666-21ssh23:34:242
22011992420,1cyclictest0-21swapper/222:59:482
22011992420,0cyclictest0-21swapper/223:45:442
2201199240,2cyclictest10949-21ssh23:10:092
2201199240,23cyclictest16029-21diskmemload23:59:412
2201199240,23cyclictest16029-21diskmemload21:53:342
2201199240,23cyclictest0-21swapper/223:37:122
2201199240,23cyclictest0-21swapper/222:07:392
2201199240,23cyclictest0-21swapper/221:38:552
2201199240,23cyclictest0-21swapper/221:16:562
2201199240,23cyclictest0-21swapper/200:05:022
22007992424,0cyclictest16029-21diskmemload00:14:171
2200799240,24cyclictest0-21swapper/122:07:121
2200799240,24cyclictest0-21swapper/100:26:491
2200799240,23cyclictest0-21swapper/123:41:561
2200799240,23cyclictest0-21swapper/121:57:121
22004992424,0cyclictest0-21swapper/021:37:430
22004992422,1cyclictest13916-21sh22:42:410
2200499241,0cyclictest13-21ksoftirqd/023:59:000
2200499240,2cyclictest5971-21ssh22:04:350
2200499240,24cyclictest0-21swapper/022:56:480
2200499240,24cyclictest0-21swapper/000:37:300
2200499240,1cyclictest0-21swapper/021:29:390
22015992323,0cyclictest16029-21diskmemload21:58:063
22015992323,0cyclictest0-21swapper/322:57:363
22015992323,0cyclictest0-21swapper/321:49:253
22015992323,0cyclictest0-21swapper/321:28:023
22015992323,0cyclictest0-21swapper/300:09:083
22015992321,1cyclictest0-21swapper/323:47:113
22015992321,1cyclictest0-21swapper/323:23:353
2201599230,23cyclictest0-21swapper/321:22:013
2201599230,22cyclictest28784-21if_eno121:54:413
2201599230,22cyclictest26408-21latency_hist20:25:013
2201599230,22cyclictest16029-21diskmemload23:40:253
2201599230,22cyclictest16029-21diskmemload22:35:063
2201599230,22cyclictest16029-21diskmemload22:27:433
2201599230,22cyclictest14438-21sh00:17:293
2201599230,22cyclictest0-21swapper/323:53:033
2201599230,22cyclictest0-21swapper/321:17:513
2201599230,1cyclictest16029-21diskmemload23:58:153
2201599230,0cyclictest16029-21diskmemload21:34:493
2201599230,0cyclictest0-21swapper/322:20:403
22011992323,0cyclictest16029-21diskmemload00:31:492
22011992323,0cyclictest0-21swapper/223:09:402
22011992323,0cyclictest0-21swapper/200:02:412
22011992321,1cyclictest0-21swapper/222:54:092
22011992320,2cyclictest606-21dbus-daemon23:05:002
22011992320,2cyclictest458-21ssh21:26:282
22011992320,2cyclictest27444-21ssh21:20:552
22011992320,2cyclictest16971-21ssh21:10:262
2201199230,2cyclictest16288-21ssh21:42:062
2201199230,22cyclictest18501-21ssh22:46:482
2201199230,22cyclictest1203-21cron20:40:002
22007992323,0cyclictest0-21swapper/123:50:081
22007992323,0cyclictest0-21swapper/123:23:391
22007992323,0cyclictest0-21swapper/122:45:361
22007992323,0cyclictest0-21swapper/122:34:391
22007992323,0cyclictest0-21swapper/122:15:081
22007992322,1cyclictest0-21swapper/122:40:351
22007992321,1cyclictest23969-21cron20:20:011
22007992321,1cyclictest23969-21cron20:20:011
22007992321,1cyclictest18734-21sh00:21:061
22007992321,1cyclictest0-21swapper/122:39:401
22007992321,1cyclictest0-21swapper/121:15:071
2200799231,21cyclictest23742-21idleruntime21:49:411
2200799230,2cyclictest12683-21sh23:12:531
2200799230,23cyclictest0-21swapper/122:00:531
2200799230,23cyclictest0-21swapper/121:53:121
2200799230,23cyclictest0-21swapper/121:42:271
2200799230,23cyclictest0-21swapper/121:36:441
2200799230,23cyclictest0-21swapper/121:29:331
2200799230,23cyclictest0-21swapper/100:33:241
2200799230,23cyclictest0-21swapper/100:17:481
2200799230,22cyclictest6535-21sh23:38:581
2200799230,22cyclictest541-21ssh23:32:451
2200799230,22cyclictest30227-21ssh00:00:101
2200799230,22cyclictest27413-21sh21:20:511
2200799230,1cyclictest29130-21ssh22:57:311
2200799230,0cyclictest16029-21diskmemload22:26:021
2200799230,0cyclictest0-21swapper/123:49:231
2200799230,0cyclictest0-21swapper/123:28:461
2200799230,0cyclictest0-21swapper/100:07:081
22004992323,0cyclictest0-21swapper/023:45:400
22004992323,0cyclictest0-21swapper/022:21:040
22004992323,0cyclictest0-21swapper/022:12:090
22004992323,0cyclictest0-21swapper/021:17:180
22004992323,0cyclictest0-21swapper/000:00:090
22004992322,1cyclictest62550irq/127-eno123:43:170
2200499232,21cyclictest62550irq/127-eno121:25:010
22004992322,1cyclictest0-21swapper/023:37:480
22004992321,1cyclictest13-21ksoftirqd/021:10:010
22004992321,1cyclictest0-21swapper/023:30:430
2200499230,23cyclictest0-21swapper/022:32:400
2200499230,22cyclictest30359-21rm21:55:050
2200499230,22cyclictest16029-21diskmemload00:30:160
2200499230,22cyclictest0-21swapper/023:05:390
2200499230,22cyclictest0-21swapper/022:52:150
2200499230,22cyclictest0-21swapper/022:17:330
2200499230,22cyclictest0-21swapper/021:34:280
2200499230,22cyclictest0-21swapper/000:21:200
2200499230,0cyclictest0-21swapper/023:27:000
2200499230,0cyclictest0-21swapper/023:01:380
2200499230,0cyclictest0-21swapper/022:35:480
2200499230,0cyclictest0-21swapper/021:14:420
22015992222,0cyclictest16029-21diskmemload23:00:053
22015992222,0cyclictest0-21swapper/323:31:253
22015992222,0cyclictest0-21swapper/322:15:123
22015992220,1cyclictest23166-21diskmemload22:51:033
22015992220,1cyclictest0-21swapper/323:05:013
22015992220,1cyclictest0-21swapper/322:11:343
22015992220,1cyclictest0-21swapper/322:05:393
2201599220,21cyclictest0-21swapper/323:25:023
2201599220,21cyclictest0-21swapper/322:30:283
2201599220,21cyclictest0-21swapper/300:38:573
2201599220,21cyclictest0-21swapper/300:23:263
2201599220,1cyclictest7221-21kworker/u8:222:45:493
2201599220,1cyclictest0-21swapper/300:11:593
2201599220,1cyclictest0-21swapper/300:00:563
2201599220,0cyclictest0-21swapper/323:10:073
22011992222,0cyclictest0-21swapper/222:16:192
22011992221,0cyclictest0-21swapper/221:46:272
22011992220,1cyclictest606-21dbus-daemon21:05:012
22011992220,1cyclictest18439-21ssh00:20:312
22011992220,1cyclictest0-21swapper/223:20:082
22011992220,1cyclictest0-21swapper/222:40:312
2201199220,2cyclictest7364-21ssh22:35:202
2201199220,22cyclictest16029-21diskmemload22:22:532
2201199220,22cyclictest0-21swapper/200:26:352
2201199220,22cyclictest0-21swapper/200:13:132
2201199220,21cyclictest9459-21apt-get20:59:392
2201199220,21cyclictest8456-21latency_hist20:55:002
2201199220,21cyclictest3297-21ssh22:00:202
2201199220,21cyclictest20525-21ssh23:51:172
2201199220,21cyclictest19809-21apt-get20:14:382
2201199220,21cyclictest16029-21diskmemload00:15:342
2201199220,21cyclictest0-21swapper/222:25:462
2201199220,1cyclictest0-21swapper/221:55:332
22007992222,0cyclictest26764-21apt-get22:24:391
22007992222,0cyclictest0-21swapper/123:55:441
22007992222,0cyclictest0-21swapper/123:16:231
22007992222,0cyclictest0-21swapper/122:52:131
22007992222,0cyclictest0-21swapper/121:30:471
22007992222,0cyclictest0-21swapper/121:11:041
22007992221,1cyclictest539-21kworker/u8:023:02:091
22007992221,1cyclictest0-21swapper/119:30:011
2200799220,22cyclictest0-21swapper/123:06:031
2200799220,21cyclictest8448-21apt-get20:55:001
2200799220,21cyclictest27379-21apt-get19:24:371
2200799220,21cyclictest0-21swapper/120:45:001
2200799220,1cyclictest313-21systemd-journal20:25:011
2200799220,1cyclictest1112-21ssh00:35:071
2200799220,0cyclictest0-21swapper/122:11:211
22004992220,1cyclictest20093-21sh21:45:040
22004992220,1cyclictest0-21swapper/019:25:010
22004992220,1cyclictest0-21swapper/000:15:520
2200499220,22cyclictest0-21swapper/023:20:380
2200499220,22cyclictest0-21swapper/022:25:150
2200499220,21cyclictest29424-21apt-get20:34:390
2200499220,21cyclictest16029-21diskmemload22:45:070
2200499220,21cyclictest0-21swapper/023:16:140
2200499220,21cyclictest0-21swapper/021:51:090
2200499220,1cyclictest3976-21sh00:06:580
2200499220,1cyclictest23353-21ssh00:25:170
2200499220,1cyclictest16029-21diskmemload22:05:200
2200499220,0cyclictest0-21swapper/000:12:520
22015992119,1cyclictest11252-21munin-run21:00:003
22015992118,2cyclictest1265-21awk20:40:003
2201599210,20cyclictest0-21swapper/320:14:383
2201599210,1cyclictest3993-21idleruntime-cro19:40:013
2201599210,0cyclictest11875-21apt-get21:04:393
2201599210,0cyclictest0-21swapper/323:35:113
2201599210,0cyclictest0-21swapper/300:30:203
22011992121,0cyclictest0-21swapper/222:10:072
22011992120,1cyclictest0-21swapper/219:39:382
22011992119,1cyclictest1286-21rm00:35:312
22011992119,1cyclictest0-21swapper/220:35:012
22007992121,0cyclictest0-21swapper/120:39:401
2200799210,21cyclictest0-21swapper/120:34:391
2200799210,1cyclictest22567-21apt-get19:14:381
2200799210,0cyclictest0-21swapper/119:45:011
22004992120,1cyclictest32176-21apt-get19:34:380
2200499210,20cyclictest0-21swapper/020:14:380
2200499210,20cyclictest0-21swapper/019:19:370
22015992020,0cyclictest0-21swapper/321:09:403
22015992020,0cyclictest0-21swapper/320:54:393
2201599200,0cyclictest0-21swapper/320:34:393
22011992020,0cyclictest0-21swapper/219:19:382
22011992019,1cyclictest1877-21apt-get20:44:382
2201199200,0cyclictest0-21swapper/219:50:002
2201199200,0cyclictest0-21swapper/219:29:382
2201199200,0cyclictest0-21swapper/219:14:382
22007992018,1cyclictest4270-21apt-get20:49:381
2200799200,0cyclictest0-21swapper/119:59:381
22004992020,0cyclictest0-21swapper/020:49:380
22004992020,0cyclictest0-21swapper/020:19:390
22004992020,0cyclictest0-21swapper/020:19:390
22004992020,0cyclictest0-21swapper/019:39:380
2200499200,0cyclictest0-21swapper/019:49:370
22015991919,0cyclictest0-21swapper/320:04:383
22015991919,0cyclictest0-21swapper/319:14:383
22015991918,1cyclictest9212-21munin-run19:50:003
22011991919,0cyclictest0-21swapper/220:09:392
22011991919,0cyclictest0-21swapper/219:24:382
22011991918,0cyclictest14624-21apt-get20:04:372
22007991919,0cyclictest0-21swapper/120:55:051
22007991919,0cyclictest0-21swapper/119:49:371
22004991919,0cyclictest0-21swapper/020:55:050
22004991919,0cyclictest0-21swapper/019:54:380
22004991918,0cyclictest0-21swapper/019:14:380
2200499190,18cyclictest0-21swapper/020:39:400
22015991818,0cyclictest0-21swapper/319:29:383
22015991817,1cyclictest29177-21grep19:25:013
22015991816,1cyclictest24963-21apt-get19:19:383
22015991816,1cyclictest17019-21apt-get20:09:393
22007991818,0cyclictest2213-21apt-get19:39:381
22007991818,0cyclictest0-21swapper/121:04:391
22007991816,1cyclictest0-21swapper/120:15:011
22004991818,0cyclictest0-21swapper/021:04:390
2200499180,1cyclictest622-21gdbus20:25:010
2200499180,17cyclictest0-21swapper/019:59:370
2200499180,15cyclictest151rcu_preempt20:29:430
22015991716,1cyclictest3989-21apt-get19:40:023
22007991717,0cyclictest0-21swapper/120:04:371
22004991716,1cyclictest13-21ksoftirqd/020:55:000
22004991716,0cyclictest0-21swapper/020:09:390
22015991614,1cyclictest23805-21sed20:19:473
22015991614,1cyclictest23805-21sed20:19:473
22011991614,1cyclictest6038-21awk20:50:002
22007991616,0cyclictest14270-21apt-get21:09:401
22004991616,0cyclictest0-21swapper/019:44:370
22004991616,0cyclictest0-21swapper/019:29:380
22015991515,0cyclictest0-21swapper/320:44:383
2201599150,0cyclictest0-21swapper/319:34:383
22011991515,0cyclictest0-21swapper/221:09:402
2201199150,15cyclictest0-21swapper/219:34:392
2201199150,14cyclictest0-21swapper/219:51:422
22007991515,0cyclictest27024-21apt-get20:29:391
22007991515,0cyclictest0-21swapper/120:09:391
22004991512,2cyclictest16419-21munin-run20:05:000
22015991414,0cyclictest0-21swapper/320:25:173
22015991414,0cyclictest0-21swapper/319:56:363
2201599140,14cyclictest0-21swapper/320:45:563
2201599140,14cyclictest0-21swapper/319:50:423
22011991413,1cyclictest0-21swapper/220:25:332
22011991411,3cyclictest0-21swapper/220:15:092
22011991411,3cyclictest0-21swapper/220:15:092
2201199140,14cyclictest0-21swapper/220:20:252
2201199140,0cyclictest0-21swapper/219:57:142
2200799140,14cyclictest0-21swapper/119:50:511
2200799140,14cyclictest0-21swapper/119:15:191
2200799140,0cyclictest0-21swapper/119:30:121
2200499140,14cyclictest0-21swapper/020:40:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional