You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-05 - 03:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Sun Apr 05, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12764993027,2cyclictest34-21ksoftirqd/323:40:013
12764992523,1cyclictest31501-21apt-get23:08:473
12760992523,1cyclictest29639-21apt-get21:33:472
1276099250,24cyclictest6402-21diskmemload23:10:032
1276099250,24cyclictest6402-21diskmemload22:05:552
1276099250,24cyclictest11083-21ssh22:17:582
1275699250,24cyclictest6402-21diskmemload21:55:051
1275499250,1cyclictest1559-21apt-get00:13:500
1276499240,23cyclictest6402-21diskmemload21:48:593
1276499240,23cyclictest0-21swapper/300:28:163
1276499240,1cyclictest25375-21ssh00:03:573
12760992423,1cyclictest6402-21diskmemload00:28:482
12760992422,1cyclictest32254-21rm22:38:192
1276099240,23cyclictest22329-21dump-pmu-power19:30:012
1276099240,23cyclictest13393-21ssh23:21:222
1276099240,0cyclictest24985-21ssh23:33:232
1275699240,24cyclictest0-21swapper/100:03:271
1275699240,23cyclictest6402-21diskmemload23:43:441
1275699240,23cyclictest4576-21apt-get21:08:481
1275699240,23cyclictest0-21swapper/121:53:021
1275699240,23cyclictest0-21swapper/100:38:491
1275699240,0cyclictest0-21swapper/121:25:391
12754992423,1cyclictest6402-21diskmemload22:59:200
12754992423,1cyclictest0-21swapper/022:20:200
12754992421,2cyclictest0-21swapper/023:23:500
1275499240,2cyclictest22337-21munin-run19:30:010
1275499240,23cyclictest3939-21sh22:40:570
1275499240,23cyclictest31111-21cron22:05:010
1275499240,23cyclictest18571-21sh22:55:010
1275499240,1cyclictest6402-21diskmemload23:38:070
12764992323,0cyclictest0-21swapper/322:35:373
12764992323,0cyclictest0-21swapper/322:02:433
12764992321,1cyclictest0-21swapper/322:28:123
12764992321,1cyclictest0-21swapper/322:24:573
1276499231,21cyclictest0-21swapper/322:16:113
1276499230,22cyclictest6402-21diskmemload23:52:113
1276499230,22cyclictest6402-21diskmemload00:32:193
1276499230,22cyclictest6402-21diskmemload00:05:203
1276499230,22cyclictest5423-21sh21:40:223
1276499230,22cyclictest29471-21runrttasks21:33:343
1276499230,22cyclictest24367-21sh20:40:013
1276499230,22cyclictest21981-21perf20:35:013
1276499230,22cyclictest13418-21turbostat.cron22:50:013
1276499230,22cyclictest13418-21turbostat.cron22:50:013
1276499230,22cyclictest12336-21ssh21:15:573
1276499230,22cyclictest0-21swapper/323:45:493
1276499230,22cyclictest0-21swapper/323:00:263
1276499230,0cyclictest6402-21diskmemload22:50:223
12760992323,0cyclictest6402-21diskmemload22:48:132
12760992323,0cyclictest6402-21diskmemload22:48:132
12760992323,0cyclictest0-21swapper/223:58:062
12760992323,0cyclictest0-21swapper/223:35:032
12760992323,0cyclictest0-21swapper/222:50:202
12760992321,2cyclictest11895-21ssh23:51:232
12760992321,1cyclictest0-21swapper/200:07:292
12760992321,1cyclictest0-21swapper/200:04:552
12760992320,2cyclictest6402-21diskmemload22:43:562
1276099230,2cyclictest29189-21basename20:50:002
1276099230,23cyclictest0-21swapper/222:25:362
1276099230,23cyclictest0-21swapper/221:15:492
1276099230,22cyclictest7587-21apt-get21:43:482
1276099230,22cyclictest609-21rs:main2
1276099230,22cyclictest22405-21sh23:29:412
1276099230,22cyclictest1913-21ssh21:38:092
1276099230,22cyclictest1166-21ssh23:09:282
1276099230,22cyclictest10694-21rm00:21:252
1276099230,0cyclictest6402-21diskmemload22:59:142
1276099230,0cyclictest0-21swapper/223:46:512
1276099230,0cyclictest0-21swapper/222:33:242
1276099230,0cyclictest0-21swapper/221:47:312
1276099230,0cyclictest0-21swapper/200:13:092
12756992323,0cyclictest6402-21diskmemload23:46:291
12756992323,0cyclictest6402-21diskmemload22:15:381
12756992323,0cyclictest0-21swapper/123:16:501
12756992323,0cyclictest0-21swapper/121:42:011
12756992323,0cyclictest0-21swapper/121:36:111
12756992323,0cyclictest0-21swapper/100:26:331
12756992323,0cyclictest0-21swapper/100:22:501
12756992321,1cyclictest28332-21ssh23:35:361
12756992321,1cyclictest25176-21sh23:02:021
12756992321,1cyclictest16803-21apt-get22:23:481
12756992321,1cyclictest0-21swapper/121:48:471
12756992320,2cyclictest22827-21sed22:58:581
1275699230,23cyclictest6402-21diskmemload23:09:301
1275699230,23cyclictest0-21swapper/122:48:201
1275699230,23cyclictest0-21swapper/122:48:201
1275699230,23cyclictest0-21swapper/122:41:561
1275699230,23cyclictest0-21swapper/122:28:131
1275699230,22cyclictest9578-21perf20:10:001
1275699230,22cyclictest6402-21diskmemload23:51:241
1275699230,22cyclictest6402-21diskmemload00:32:251
1275699230,22cyclictest4195-21ssh22:10:101
1275699230,22cyclictest26433-21sh22:00:281
1275699230,0cyclictest6402-21diskmemload22:07:161
1275699230,0cyclictest0-21swapper/123:59:051
12754992323,0cyclictest0-21swapper/023:12:550
12754992323,0cyclictest0-21swapper/022:36:480
12754992323,0cyclictest0-21swapper/022:31:480
12754992321,1cyclictest9545-21cron20:10:000
12754992321,1cyclictest0-21swapper/023:00:040
12754992321,1cyclictest0-21swapper/022:25:490
12754992321,1cyclictest0-21swapper/000:30:470
12754992320,2cyclictest1275-21/usr/sbin/munin22:08:560
1275499230,2cyclictest7345-21ssh21:43:280
1275499230,2cyclictest11612-21rm00:22:540
1275499230,23cyclictest0-21swapper/021:26:420
1275499230,22cyclictest6523-21ssh23:45:580
1275499230,22cyclictest6430-21awk21:10:010
1275499230,22cyclictest6402-21diskmemload23:30:510
1275499230,22cyclictest622-21gdbus19:40:010
1275499230,22cyclictest3609-21ssh23:43:510
1275499230,22cyclictest17675-21sh23:25:020
1275499230,22cyclictest17546-21awk19:20:000
1275499230,22cyclictest14133-21cron22:20:000
1275499230,22cyclictest0-21swapper/021:48:370
1275499230,1cyclictest0-21swapper/023:54:390
1275499230,1cyclictest0-21swapper/023:17:180
1275499230,1cyclictest0-21swapper/021:57:320
1275499230,1cyclictest0-21swapper/000:07:340
1275499230,0cyclictest0-21swapper/021:37:050
1275499230,0cyclictest0-21swapper/000:04:000
12764992222,0cyclictest0-21swapper/322:05:323
12764992221,1cyclictest6402-21diskmemload22:56:263
12764992220,1cyclictest26787-21perf20:45:003
12764992220,1cyclictest20678-21ssh21:55:073
12764992220,1cyclictest19725-21sh21:53:573
12764992220,1cyclictest18564-21ssh23:26:283
12764992220,1cyclictest0-21swapper/322:30:513
12764992220,1cyclictest0-21swapper/300:20:353
1276499220,2cyclictest8102-21sh23:16:073
1276499220,21cyclictest6402-21diskmemload21:25:213
1276499220,21cyclictest5201-21ssh00:15:523
1276499220,21cyclictest4313-21ssh22:41:343
1276499220,21cyclictest32444-21ssh21:35:063
1276499220,21cyclictest2730-21ssh23:10:493
1276499220,21cyclictest14790-21sendmail-msp20:20:013
1276499220,1cyclictest23959-21sh23:31:543
1276499220,1cyclictest18431-21ssh23:58:403
1276499220,1cyclictest0-21swapper/323:21:253
1276499220,1cyclictest0-21swapper/321:13:323
12760992222,0cyclictest6402-21diskmemload00:31:022
12760992222,0cyclictest0-21swapper/221:53:002
12760992222,0cyclictest0-21swapper/200:17:422
12760992221,1cyclictest28-21ksoftirqd/222:22:492
12760992220,1cyclictest8033-21ssh23:16:012
12760992220,1cyclictest0-21swapper/223:01:402
12760992220,1cyclictest0-21swapper/221:56:052
12760992220,1cyclictest0-21swapper/221:25:502
1276099220,21cyclictest6402-21diskmemload21:11:522
1276099220,21cyclictest26838-21ssh22:01:022
1276099220,21cyclictest0-21swapper/220:33:462
1276099220,21cyclictest0-21swapper/220:10:002
1276099220,1cyclictest5721-21ssh22:12:332
1276099220,1cyclictest26823-21ssh00:36:542
1276099220,0cyclictest6402-21diskmemload23:40:422
12756992222,0cyclictest0-21swapper/122:33:121
12756992222,0cyclictest0-21swapper/100:13:501
12756992221,1cyclictest0-21swapper/121:20:031
12756992219,2cyclictest4638-21sh00:15:031
1275699220,22cyclictest0-21swapper/123:28:171
1275699220,21cyclictest6402-21diskmemload22:50:201
1275699220,21cyclictest23927-21ssh23:31:511
1275699220,21cyclictest13420-21sh23:21:251
1275699220,21cyclictest0-21swapper/122:35:051
1275699220,1cyclictest6402-21diskmemload21:30:071
1275699220,1cyclictest27531-21apt19:40:011
1275699220,1cyclictest19567-21idleruntime-cro20:30:011
1275699220,1cyclictest0-21swapper/123:10:241
12754992222,0cyclictest0-21swapper/000:25:570
12754992221,1cyclictest0-21swapper/021:30:430
12754992221,1cyclictest0-21swapper/021:16:150
12754992220,1cyclictest7159-21cron20:05:000
12754992220,1cyclictest30516-21cron00:40:000
1275499220,21cyclictest6402-21diskmemload21:20:400
1275499220,21cyclictest29414-21ssh23:05:270
1275499220,21cyclictest20990-21cron00:00:000
1275499220,21cyclictest16142-21ssh21:51:090
1275499220,21cyclictest0-21swapper/022:47:250
1275499220,21cyclictest0-21swapper/022:47:250
1275499220,21cyclictest0-21swapper/022:10:360
1275499220,21cyclictest0-21swapper/020:13:470
1275499220,1cyclictest24344-21cron20:40:010
12764992121,0cyclictest0-21swapper/320:23:483
12764992120,1cyclictest26217-21ssh00:35:523
1276499210,21cyclictest0-21swapper/300:10:093
1276499210,1cyclictest19320-21apt-get21:23:493
1276499210,1cyclictest17528-21grep19:20:003
1276499210,0cyclictest0-21swapper/323:40:043
12760992121,0cyclictest32123-21apt-get20:58:482
12760992120,1cyclictest2129-21apt-get21:03:482
12760992119,0cyclictest151rcu_preempt19:48:472
1276099210,1cyclictest18087-21apt-get19:23:482
1276099210,0cyclictest0-21swapper/221:08:482
12756992121,0cyclictest0-21swapper/121:10:551
1275699210,0cyclictest0-21swapper/120:43:481
12754992121,0cyclictest0-21swapper/021:11:040
12754992120,0cyclictest0-21swapper/019:58:490
12754992119,1cyclictest31572-21dump-pmu-power20:55:000
12764992020,0cyclictest5327-21apt-get20:03:473
12764992020,0cyclictest0-21swapper/319:53:483
1276499200,20cyclictest0-21swapper/321:03:483
1276499200,0cyclictest0-21swapper/319:13:463
12760992020,0cyclictest28082-21apt-get19:43:472
12760992020,0cyclictest10512-21apt-get20:13:472
12760992020,0cyclictest0-21swapper/220:38:482
1276099200,0cyclictest0-21swapper/220:03:472
12756992020,0cyclictest0-21swapper/121:03:481
12756992020,0cyclictest0-21swapper/119:18:461
12756992019,1cyclictest0-21swapper/119:43:471
12756992017,2cyclictest15341-21chrt20:23:481
12754992020,0cyclictest0-21swapper/020:58:480
12764991919,0cyclictest0-21swapper/320:53:493
12764991919,0cyclictest0-21swapper/320:28:473
12764991917,1cyclictest606-21dbus-daemon19:30:013
1276499190,0cyclictest22877-21apt-get19:33:473
12760991919,0cyclictest17729-21apt-get20:28:472
12756991919,0cyclictest0-21swapper/120:58:481
12756991919,0cyclictest0-21swapper/119:28:471
12756991918,1cyclictest9569-21apt-get20:10:021
12756991918,1cyclictest0-21swapper/119:33:471
12754991919,0cyclictest0-21swapper/020:23:480
12754991918,1cyclictest3970-21turbostat21:05:000
12764991818,0cyclictest0-21swapper/319:48:473
12760991817,0cyclictest0-21swapper/219:38:462
12760991816,1cyclictest17527-21cat19:20:002
1276099180,1cyclictest151rcu_preempt19:58:512
12756991818,0cyclictest0-21swapper/120:33:461
12756991818,0cyclictest0-21swapper/119:53:481
12756991817,1cyclictest32325-21perf19:50:011
12754991818,0cyclictest0-21swapper/020:18:480
12754991817,1cyclictest0-21swapper/019:53:480
12754991816,1cyclictest0-21swapper/019:35:010
12764991716,1cyclictest34-21ksoftirqd/319:25:003
12760991717,0cyclictest0-21swapper/220:18:492
12754991717,0cyclictest0-21swapper/019:43:470
12764991615,1cyclictest0-21swapper/320:10:023
12764991613,2cyclictest6435-21perf21:10:013
1276499160,15cyclictest0-21swapper/320:58:483
12760991613,2cyclictest26789-21cat20:45:002
12756991616,0cyclictest0-21swapper/120:38:481
12756991616,0cyclictest0-21swapper/119:23:481
1275699160,2cyclictest14783-21perf20:20:011
12754991616,0cyclictest0-21swapper/019:25:000
12754991614,1cyclictest0-21swapper/020:28:540
1275499160,16cyclictest13271-21apt-get19:13:460
12764991515,0cyclictest0-21swapper/319:43:473
12764991515,0cyclictest0-21swapper/319:38:463
1276099150,1cyclictest15102-21awk19:15:002
12756991515,0cyclictest0-21swapper/120:03:471
12754991515,0cyclictest20123-21apt-get20:33:460
1276499140,14cyclictest0-21swapper/320:46:053
1276499140,14cyclictest0-21swapper/320:05:233
1276499140,14cyclictest0-21swapper/319:56:103
12760991414,0cyclictest0-21swapper/220:20:272
12760991414,0cyclictest0-21swapper/219:30:092
1276099140,14cyclictest0-21swapper/219:50:172
1275699140,0cyclictest0-21swapper/120:50:081
1275699140,0cyclictest0-21swapper/120:45:101
1275699140,0cyclictest0-21swapper/119:56:301
1275499140,14cyclictest0-21swapper/020:40:150
1275499140,0cyclictest0-21swapper/020:45:080
1275499140,0cyclictest0-21swapper/019:46:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional