You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-16 - 13:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Jun 16, 2026 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1669799270,23cyclictest0-21swapper/122:07:041
1669999260,2cyclictest16308-21ssh00:19:013
1669899260,2cyclictest9969-21diskmemload23:27:392
1669899260,25cyclictest6432-21ssh23:09:172
16697992621,4cyclictest22261-21apt-get23:54:491
16697992619,3cyclictest0-21swapper/123:38:111
1669799260,25cyclictest0-21swapper/123:00:001
1669799260,25cyclictest0-21swapper/122:24:151
1669799260,1cyclictest0-21swapper/123:18:081
16699992524,0cyclictest24909-21apt-get19:29:473
16698992522,2cyclictest0-21swapper/222:54:212
16697992522,3cyclictest0-21swapper/122:38:131
16697992520,4cyclictest23303-21apt-get22:54:501
1669799251,23cyclictest8336-21ssh21:39:501
1669799250,24cyclictest0-21swapper/123:46:111
1669799250,24cyclictest0-21swapper/123:01:281
1669799250,24cyclictest0-21swapper/100:21:431
1669799250,21cyclictest9969-21diskmemload23:30:111
1669799250,21cyclictest9969-21diskmemload21:15:151
1669799250,20cyclictest0-21swapper/122:01:091
1669799250,0cyclictest0-21swapper/121:13:231
1669699250,24cyclictest21316-21ssh23:22:310
1669999240,23cyclictest9969-21diskmemload21:57:283
1669999240,23cyclictest22750-21ssh21:53:073
1669999240,1cyclictest4609-21rm22:35:363
16698992423,1cyclictest9969-21diskmemload23:37:172
16698992421,2cyclictest28518-21ssh22:28:512
1669899241,2cyclictest31979-21ssh00:33:282
1669899241,23cyclictest0-21swapper/222:33:342
1669899241,22cyclictest9969-21diskmemload22:15:012
1669899241,22cyclictest29877-21ssh00:00:072
1669899240,24cyclictest0-21swapper/222:14:582
1669899240,24cyclictest0-21swapper/221:22:132
1669899240,23cyclictest0-21swapper/223:47:412
1669899240,23cyclictest0-21swapper/222:45:182
1669899240,1cyclictest10184-21ssh22:41:012
1669899240,0cyclictest9969-21diskmemload21:39:022
16697992421,2cyclictest1-21systemd22:45:011
1669799240,24cyclictest0-21swapper/122:47:221
1669799240,24cyclictest0-21swapper/100:28:001
1669799240,23cyclictest12040-21ssh21:43:001
1669799240,23cyclictest0-21swapper/122:16:221
1669799240,21cyclictest0-21swapper/122:25:391
1669799240,0cyclictest9969-21diskmemload23:44:201
1669799240,0cyclictest9969-21diskmemload00:31:481
1669799240,0cyclictest0-21swapper/122:30:131
1669699240,24cyclictest0-21swapper/022:58:590
1669699240,24cyclictest0-21swapper/021:27:300
1669699240,23cyclictest9969-21diskmemload21:21:370
1669699240,23cyclictest0-21swapper/000:22:070
1669699240,1cyclictest9969-21diskmemload22:21:330
1669699240,1cyclictest2901-21ssh00:05:170
1669699240,0cyclictest9969-21diskmemload23:29:180
1669699240,0cyclictest9969-21diskmemload22:10:590
16699992323,0cyclictest0-21swapper/323:30:263
16699992323,0cyclictest0-21swapper/323:23:283
16699992323,0cyclictest0-21swapper/300:39:173
16699992321,1cyclictest0-21swapper/322:31:513
16699992320,2cyclictest27960-21sendmail20:40:003
16699992320,2cyclictest2153-21ssh21:34:153
1669999230,23cyclictest0-21swapper/321:48:033
1669999230,22cyclictest9803-21sh23:11:183
1669999230,22cyclictest8444-21ssh00:10:083
1669999230,22cyclictest8444-21ssh00:10:083
1669999230,22cyclictest30369-21ssh00:31:003
1669999230,22cyclictest25746-21sh23:26:103
1669999230,22cyclictest14618-21idleruntime-cro22:45:013
1669999230,22cyclictest0-21swapper/323:16:253
1669999230,22cyclictest0-21swapper/321:20:063
1669999230,22cyclictest0-21swapper/319:24:493
1669999230,0cyclictest9969-21diskmemload23:57:043
1669999230,0cyclictest9969-21diskmemload23:38:403
1669999230,0cyclictest0-21swapper/322:58:583
1669999230,0cyclictest0-21swapper/322:53:353
16698992323,0cyclictest0-21swapper/222:21:432
16698992323,0cyclictest0-21swapper/200:21:592
16698992322,1cyclictest31472-21perf19:40:002
16698992322,1cyclictest0-21swapper/223:55:522
16698992321,1cyclictest0-21swapper/223:31:252
16698992321,1cyclictest0-21swapper/223:15:312
16698992321,1cyclictest0-21swapper/221:50:482
16698992321,1cyclictest0-21swapper/219:55:002
16698992321,1cyclictest0-21swapper/200:35:132
16698992320,2cyclictest3229-21rm00:05:462
16698992320,2cyclictest13209-21grep00:14:582
16698992320,2cyclictest13209-21grep00:14:582
1669899231,21cyclictest27245-21ssh21:56:562
1669899231,21cyclictest0-21swapper/222:07:202
1669899230,23cyclictest0-21swapper/222:58:302
1669899230,23cyclictest0-21swapper/221:34:272
1669899230,22cyclictest9026-21ssh23:40:492
1669899230,22cyclictest10265-21ssh21:40:232
1669899230,22cyclictest0-21swapper/223:10:292
1669899230,22cyclictest0-21swapper/221:47:422
16697992323,0cyclictest0-21swapper/123:05:351
16697992323,0cyclictest0-21swapper/121:51:211
16697992322,1cyclictest27869-21ssh23:29:231
16697992321,1cyclictest28577-21apt-get20:44:471
16697992320,3cyclictest0-21swapper/121:33:151
16697992320,2cyclictest31131-21latency_hist22:00:011
1669799230,23cyclictest0-21swapper/123:21:251
1669799230,23cyclictest0-21swapper/122:10:411
1669799230,23cyclictest0-21swapper/121:25:571
1669799230,23cyclictest0-21swapper/121:21:161
1669799230,22cyclictest7732-21apt-get00:10:021
1669799230,22cyclictest7732-21apt-get00:10:021
1669799230,22cyclictest27915-21cron20:40:001
1669799230,22cyclictest15587-21ssh21:45:181
1669799230,22cyclictest0-21swapper/100:35:141
1669799230,1cyclictest31466-21grep19:40:001
1669799230,10cyclictest18975-21apt-get20:24:481
1669799230,0cyclictest0-21swapper/123:55:321
1669799230,0cyclictest0-21swapper/123:10:131
16696992323,0cyclictest0-21swapper/000:00:100
16696992322,1cyclictest9969-21diskmemload23:00:540
1669699232,21cyclictest62550irq/127-eno100:18:470
16696992322,1cyclictest30899-21ssh00:31:510
16696992321,1cyclictest15973-21ssh22:46:440
16696992321,1cyclictest0-21swapper/023:57:210
16696992321,1cyclictest0-21swapper/023:36:520
16696992320,2cyclictest5038-21ssh22:05:290
16696992320,2cyclictest32216-21sh22:31:290
1669699230,2cyclictest14617-21awk22:45:010
1669699230,23cyclictest11788-21ssh21:11:520
1669699230,23cyclictest0-21swapper/023:10:230
1669699230,23cyclictest0-21swapper/021:58:030
1669699230,23cyclictest0-21swapper/021:43:000
1669699230,23cyclictest0-21swapper/000:10:460
1669699230,23cyclictest0-21swapper/000:10:460
1669699230,22cyclictest6269-21sh23:09:030
1669699230,22cyclictest3578-21memory21:34:540
1669699230,22cyclictest24739-21sh00:25:370
1669699230,1cyclictest606-21dbus-daemon19:15:010
1669699230,0cyclictest9969-21diskmemload23:31:030
1669699230,0cyclictest0-21swapper/022:00:470
16699992222,0cyclictest9969-21diskmemload21:13:253
16699992222,0cyclictest0-21swapper/322:49:073
16699992222,0cyclictest0-21swapper/322:06:393
16699992222,0cyclictest0-21swapper/300:29:373
16699992221,1cyclictest10441-21ssh23:42:543
16699992219,2cyclictest10820-21ssh21:41:093
1669999220,21cyclictest9969-21diskmemload21:16:293
1669999220,21cyclictest9969-21diskmemload00:22:283
1669999220,21cyclictest9969-21diskmemload00:05:243
1669999220,21cyclictest3794-21rm23:05:093
1669999220,21cyclictest30222-21sh00:00:363
1669999220,21cyclictest16640-21sh22:17:013
1669999220,21cyclictest0-21swapper/323:53:563
1669999220,1cyclictest20631-21ssh22:20:013
1669999220,1cyclictest0-21swapper/323:03:123
1669999220,1cyclictest0-21swapper/322:12:103
1669999220,1cyclictest0-21swapper/321:27:193
1669999220,0cyclictest0-21swapper/323:45:033
1669999220,0cyclictest0-21swapper/321:36:423
16698992221,1cyclictest0-21swapper/200:25:222
16698992220,1cyclictest21466-21sendmail19:20:012
16698992220,1cyclictest0-21swapper/221:26:172
16698992220,1cyclictest0-21swapper/220:39:492
16698992220,1cyclictest0-21swapper/200:16:482
16698992219,2cyclictest30347-21idleruntime-cro20:45:002
1669899220,22cyclictest0-21swapper/222:35:322
1669899220,21cyclictest16032-21ssh21:15:162
1669899220,21cyclictest0-21swapper/223:01:382
1669899220,1cyclictest0-21swapper/223:50:382
16697992221,1cyclictest0-21swapper/100:15:321
1669799220,21cyclictest27310-21apt-get19:34:481
1669799220,21cyclictest0-21swapper/121:09:491
1669799220,1cyclictest945-21apt-get20:54:491
1669799220,1cyclictest1454-21turbostat19:45:001
16696992222,0cyclictest9969-21diskmemload22:28:300
16696992222,0cyclictest0-21swapper/022:50:370
16696992222,0cyclictest0-21swapper/000:36:530
16696992220,1cyclictest15391-21ssh23:16:460
1669699220,2cyclictest5032-21diskmemload22:36:200
1669699220,22cyclictest0-21swapper/022:18:430
1669699220,21cyclictest10007-21awk21:10:010
1669699220,1cyclictest8528-21ssh23:40:080
1669699220,1cyclictest22517-21apt-get19:24:490
1669699220,1cyclictest21366-21apt-get20:29:480
1669699220,1cyclictest16247-21ssh21:15:400
1669699220,1cyclictest0-21swapper/023:50:470
1669699220,0cyclictest0-21swapper/023:45:460
1669699220,0cyclictest0-21swapper/021:50:090
1669699220,0cyclictest0-21swapper/021:36:130
16699992120,0cyclictest9366-21apt-get20:04:513
16699992120,0cyclictest9366-21apt-get20:04:513
16699992119,2cyclictest31496-21tr19:40:003
1669999210,20cyclictest0-21swapper/319:19:493
1669999210,1cyclictest27507-21ssh22:27:203
1669999210,0cyclictest0-21swapper/320:34:503
16698992121,0cyclictest9969-21diskmemload22:00:012
16698992119,1cyclictest23129-21idleruntime-cro20:30:002
16698992119,1cyclictest16551-21apt-get20:19:482
1669899210,20cyclictest0-21swapper/220:34:502
1669899210,20cyclictest0-21swapper/220:04:512
1669899210,20cyclictest0-21swapper/220:04:512
1669899210,1cyclictest0-21swapper/220:24:492
1669899210,0cyclictest28-21ksoftirqd/220:11:202
16697992120,1cyclictest3402-21apt-get20:59:481
1669699210,20cyclictest4553-21apt-get19:54:490
1669699210,20cyclictest26160-21apt-get20:39:490
1669699210,20cyclictest0-21swapper/020:19:480
16699992020,0cyclictest0-21swapper/320:14:483
16699992020,0cyclictest0-21swapper/319:54:493
16699992020,0cyclictest0-21swapper/319:34:483
1669999200,20cyclictest3928-21awk19:50:003
1669999200,20cyclictest0-21swapper/320:29:483
16698992020,0cyclictest0-21swapper/220:09:492
16698992020,0cyclictest0-21swapper/219:29:472
1669899200,0cyclictest0-21swapper/221:04:492
1669899200,0cyclictest0-21swapper/219:44:482
16697992020,0cyclictest0-21swapper/120:19:491
16697992020,0cyclictest0-21swapper/119:54:491
16697992020,0cyclictest0-21swapper/119:24:491
16697992019,1cyclictest14152-21apt-get20:14:491
16696992020,0cyclictest0-21swapper/020:34:510
16696992020,0cyclictest0-21swapper/020:04:510
16696992020,0cyclictest0-21swapper/020:04:510
16696992019,1cyclictest5821-21apt-get21:04:490
16699991919,0cyclictest0-21swapper/320:54:483
16699991919,0cyclictest0-21swapper/319:14:493
16699991917,1cyclictest32729-21apt20:50:003
1669999190,18cyclictest0-21swapper/320:44:473
16698991919,0cyclictest0-21swapper/220:54:492
16698991914,4cyclictest0-21swapper/219:24:492
16697991919,0cyclictest0-21swapper/120:29:481
16697991918,1cyclictest0-21swapper/119:49:481
16697991916,2cyclictest7589-21cron21:05:011
16696991919,0cyclictest0-21swapper/020:54:490
16696991919,0cyclictest0-21swapper/020:44:470
16696991919,0cyclictest0-21swapper/020:14:490
16699991818,0cyclictest0-21swapper/320:19:493
16699991817,1cyclictest8216-21apt-get21:09:493
16699991816,1cyclictest5176-21perf21:00:013
16698991816,1cyclictest600-21systemd-logind21:09:562
16698991815,2cyclictest0-21swapper/220:49:472
16697991816,1cyclictest8708-21cron20:00:011
16696991815,2cyclictest20744-21latency_hist20:25:000
16699991717,0cyclictest0-21swapper/320:09:503
16699991717,0cyclictest0-21swapper/319:44:483
16698991717,0cyclictest0-21swapper/219:34:492
16698991714,2cyclictest0-21swapper/219:14:532
16697991715,1cyclictest25542-21latency_hist20:35:001
16696991717,0cyclictest0-21swapper/019:39:460
16696991715,1cyclictest21477-21perf19:20:010
16696991714,2cyclictest13527-21idleruntime-cro20:10:010
16699991616,0cyclictest0-21swapper/321:04:493
16698991616,0cyclictest0-21swapper/219:49:482
1669799160,16cyclictest0-21swapper/120:05:151
1669799160,15cyclictest0-21swapper/119:19:491
1669799160,0cyclictest0-21swapper/119:12:491
16696991614,1cyclictest0-21swapper/019:59:490
16698991514,1cyclictest0-21swapper/219:59:492
16697991513,1cyclictest609-21rs:main1
16696991515,0cyclictest0-21swapper/019:44:480
1669999140,14cyclictest0-21swapper/320:20:303
1669999140,14cyclictest0-21swapper/319:55:083
1669899140,0cyclictest0-21swapper/220:55:592
16696991414,0cyclictest0-21swapper/019:25:250
1669699140,14cyclictest0-21swapper/020:55:040
1669699140,14cyclictest0-21swapper/019:30:340
1669699140,0cyclictest0-21swapper/020:45:240
1669699140,0cyclictest0-21swapper/019:45:060
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional