You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 14:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Jul 14, 2026 00:44:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
536399260,25cyclictest0-21swapper/222:24:332
5363992523,1cyclictest0-21swapper/221:15:352
536399250,24cyclictest10935-21ssh21:21:122
536399250,1cyclictest23806-21sh22:30:002
5357992523,1cyclictest0-21swapper/121:54:351
535799250,24cyclictest0-21swapper/123:55:241
535799250,24cyclictest0-21swapper/122:31:081
5353992523,1cyclictest3562-21ssh22:40:590
535399250,24cyclictest0-21swapper/022:30:090
536799240,1cyclictest31029-21diskmemload23:24:593
536799240,1cyclictest19007-21ssh21:27:173
5363992418,2cyclictest30198-21rm00:31:282
536399240,23cyclictest0-21swapper/223:33:212
536399240,23cyclictest0-21swapper/223:16:002
5357992422,1cyclictest31704-21grep22:37:091
535799240,24cyclictest0-21swapper/121:10:191
535799240,24cyclictest0-21swapper/100:21:431
535799240,24cyclictest0-21swapper/100:09:061
535799240,23cyclictest0-21swapper/122:01:481
535799240,23cyclictest0-21swapper/121:58:081
535799240,23cyclictest0-21swapper/121:36:021
535799240,0cyclictest0-21swapper/122:44:361
535799240,0cyclictest0-21swapper/122:07:381
535799240,0cyclictest0-21swapper/121:44:191
5353992423,1cyclictest0-21swapper/023:01:180
535399240,24cyclictest0-21swapper/022:23:530
535399240,23cyclictest6906-21latency_hist22:15:000
535399240,23cyclictest25157-21/usr/sbin/munin00:27:090
535399240,0cyclictest0-21swapper/022:48:100
5367992323,0cyclictest0-21swapper/323:51:453
5367992323,0cyclictest0-21swapper/322:55:203
5367992323,0cyclictest0-21swapper/321:47:343
5367992323,0cyclictest0-21swapper/300:34:403
5367992323,0cyclictest0-21swapper/300:25:123
5367992321,1cyclictest13909-21ssh00:17:063
536799230,2cyclictest30058-21cat23:32:163
536799230,23cyclictest31029-21diskmemload22:25:173
536799230,23cyclictest0-21swapper/323:17:313
536799230,23cyclictest0-21swapper/322:45:453
536799230,23cyclictest0-21swapper/322:00:483
536799230,23cyclictest0-21swapper/321:10:113
536799230,22cyclictest31902-21ssh23:05:573
536799230,22cyclictest16875-21sh22:23:143
536799230,1cyclictest0-21swapper/323:46:423
536799230,1cyclictest0-21swapper/321:20:003
536799230,0cyclictest0-21swapper/322:41:323
536799230,0cyclictest0-21swapper/321:34:403
536799230,0cyclictest0-21swapper/300:24:283
5363992323,0cyclictest31029-21diskmemload21:27:062
5363992323,0cyclictest0-21swapper/200:35:112
5363992321,1cyclictest4160-21sh23:09:542
5363992321,1cyclictest24327-21apt-get22:02:062
5363992321,1cyclictest0-21swapper/223:21:542
5363992321,1cyclictest0-21swapper/221:50:072
5363992321,1cyclictest0-21swapper/200:10:342
5363992320,2cyclictest17201-21ssh23:50:292
5363992320,2cyclictest13905-21/usr/sbin/munin00:17:162
536399230,2cyclictest26905-21rm23:58:482
536399230,23cyclictest0-21swapper/200:20:542
536399230,22cyclictest8224-21tail21:47:092
536399230,22cyclictest31703-21ssh21:10:332
536399230,22cyclictest28604-21ssh23:02:182
536399230,22cyclictest21631-21diskstats22:57:092
536399230,22cyclictest2152-21cron20:10:002
536399230,22cyclictest20221-21apt-get19:42:072
536399230,22cyclictest12069-21ssh22:47:372
536399230,22cyclictest0-21swapper/222:05:102
536399230,1cyclictest24318-21ssh00:26:032
536399230,0cyclictest31029-21diskmemload22:52:022
536399230,0cyclictest0-21swapper/223:38:072
536399230,0cyclictest0-21swapper/200:02:272
5357992323,0cyclictest0-21swapper/123:30:101
5357992323,0cyclictest0-21swapper/123:26:111
5357992323,0cyclictest0-21swapper/122:16:221
5357992323,0cyclictest0-21swapper/121:25:091
5357992322,0cyclictest0-21swapper/121:20:331
5357992321,2cyclictest4376-21ssh00:37:091
5357992321,1cyclictest0-21swapper/123:00:041
5357992321,1cyclictest0-21swapper/122:46:111
5357992321,1cyclictest0-21swapper/121:45:591
5357992321,1cyclictest0-21swapper/100:00:471
535799230,2cyclictest16666-21munin-run20:40:001
535799230,22cyclictest31293-21ssh23:05:011
535799230,22cyclictest0-21swapper/123:49:531
535799230,1cyclictest0-21swapper/123:41:141
535799230,1cyclictest0-21swapper/100:17:061
535799230,0cyclictest0-21swapper/123:20:261
535799230,0cyclictest0-21swapper/122:22:321
5353992323,0cyclictest0-21swapper/022:29:080
5353992323,0cyclictest0-21swapper/021:57:220
5353992323,0cyclictest0-21swapper/000:30:130
5353992323,0cyclictest0-21swapper/000:15:020
5353992322,1cyclictest28894-21ssh00:01:080
5353992322,1cyclictest0-21swapper/022:01:180
5353992321,1cyclictest29806-21sendmail-msp20:00:000
5353992321,1cyclictest0-21swapper/023:53:530
5353992321,1cyclictest0-21swapper/021:53:320
5353992320,2cyclictest28233-21ssh21:36:540
5353992320,2cyclictest22185-21diskmemload23:26:190
535399231,21cyclictest31306-21rm21:10:030
535399231,1cyclictest21012-21cron21:30:000
535399230,23cyclictest0-21swapper/023:48:480
535399230,23cyclictest0-21swapper/023:41:330
535399230,23cyclictest0-21swapper/023:13:240
535399230,23cyclictest0-21swapper/022:54:180
535399230,22cyclictest7960-21sh22:16:130
535399230,22cyclictest31029-21diskmemload23:31:380
535399230,22cyclictest31029-21diskmemload21:16:260
535399230,22cyclictest19116-21ssh00:21:330
535399230,22cyclictest10032-21ssh23:15:030
535399230,22cyclictest0-21swapper/023:35:330
535399230,22cyclictest0-21swapper/000:07:540
535399230,21cyclictest0-21swapper/021:45:260
535399230,0cyclictest31029-21diskmemload23:58:410
535399230,0cyclictest0-21swapper/022:08:240
535399230,0cyclictest0-21swapper/021:20:220
5367992222,0cyclictest0-21swapper/323:41:343
5367992222,0cyclictest0-21swapper/323:37:053
5367992222,0cyclictest0-21swapper/322:52:433
5367992222,0cyclictest0-21swapper/322:10:043
5367992222,0cyclictest0-21swapper/321:56:513
5367992221,1cyclictest22704-21ssh23:55:243
5367992221,0cyclictest0-21swapper/300:07:103
536799220,22cyclictest31029-21diskmemload23:04:563
536799220,22cyclictest0-21swapper/300:01:193
536799220,21cyclictest9811-21apt-get20:27:073
536799220,21cyclictest5397-21ssh23:11:163
536799220,21cyclictest29934-21ssh22:35:323
536799220,21cyclictest29777-21dump-pmu-power20:00:003
536799220,21cyclictest0-21swapper/300:10:383
536799220,1cyclictest27681-21ssh21:36:113
536799220,1cyclictest26640-21rm23:30:013
536799220,1cyclictest24238-21ssh22:30:153
536799220,1cyclictest0-21swapper/300:35:353
536799220,0cyclictest31029-21diskmemload22:05:103
5363992222,0cyclictest31029-21diskmemload22:13:472
5363992222,0cyclictest0-21swapper/222:35:412
5363992222,0cyclictest0-21swapper/221:55:162
5363992220,1cyclictest19471-21apt-get20:47:082
5363992220,1cyclictest0-21swapper/223:42:092
5363992220,1cyclictest0-21swapper/223:10:152
5363992220,1cyclictest0-21swapper/221:40:462
5363992220,1cyclictest0-21swapper/221:35:022
536399220,22cyclictest0-21swapper/223:48:212
536399220,22cyclictest0-21swapper/222:30:242
536399220,22cyclictest0-21swapper/200:09:362
536399220,21cyclictest7258-21diskmemload22:15:082
536399220,21cyclictest31029-21diskmemload22:40:592
536399220,21cyclictest31029-21diskmemload21:33:182
536399220,21cyclictest12955-21apt-get19:27:062
5357992222,0cyclictest0-21swapper/123:51:471
5357992222,0cyclictest0-21swapper/122:58:491
5357992221,1cyclictest0-21swapper/121:15:261
5357992220,1cyclictest0-21swapper/123:15:101
5357992220,1cyclictest0-21swapper/121:32:071
5357992220,1cyclictest0-21swapper/120:05:001
535799221,1cyclictest6976-21turbostat.cron20:20:011
535799220,21cyclictest7705-21idleruntime-cro19:15:001
535799220,21cyclictest0-21swapper/122:51:391
535799220,21cyclictest0-21swapper/120:52:081
535799220,21cyclictest0-21swapper/100:10:071
535799220,1cyclictest29419-21sh00:30:211
535799220,1cyclictest27161-21diskmemload00:27:351
535799220,1cyclictest0-21swapper/123:35:011
535799220,1cyclictest0-21swapper/123:11:591
535799220,1cyclictest0-21swapper/122:11:241
535799220,1cyclictest0-21swapper/119:45:021
535799220,0cyclictest0-21swapper/122:28:591
5353992222,0cyclictest0-21swapper/023:21:280
5353992222,0cyclictest0-21swapper/021:31:300
5353992222,0cyclictest0-21swapper/000:36:290
5353992220,1cyclictest19789-21tr19:40:010
535399220,22cyclictest0-21swapper/021:41:090
535399220,22cyclictest0-21swapper/000:10:170
535399220,1cyclictest9409-21latency_hist20:25:010
535399220,1cyclictest31029-21diskmemload22:37:070
535399220,1cyclictest26243-21apt21:00:010
535399220,1cyclictest23855-21munin-run20:55:000
5367992121,0cyclictest32614-21apt-get20:07:083
5367992121,0cyclictest0-21swapper/320:52:083
536799210,21cyclictest0-21swapper/322:15:273
536799210,21cyclictest0-21swapper/321:50:063
536799210,20cyclictest5747-21apt-get19:12:053
536799210,20cyclictest0-21swapper/319:17:073
536799210,0cyclictest25407-21apt-get19:52:073
536799210,0cyclictest0-21swapper/321:02:083
536799210,0cyclictest0-21swapper/319:22:073
5363992121,0cyclictest0-21swapper/219:22:072
5363992119,1cyclictest7000-21tr20:20:012
536399210,20cyclictest0-21swapper/220:37:052
536399210,1cyclictest10151-21sendmail-msp19:20:002
536399210,0cyclictest0-21swapper/221:07:072
5357992120,1cyclictest0-21swapper/120:47:081
5357992119,1cyclictest27389-21apt19:55:001
535799210,1cyclictest0-21swapper/120:10:001
5353992121,0cyclictest0-21swapper/020:17:080
5353992120,1cyclictest19644-21ssh22:55:050
5353992120,1cyclictest13508-21kworker/u8:023:05:080
535399210,20cyclictest0-21swapper/019:42:070
5367992019,0cyclictest0-21swapper/319:42:073
536799200,20cyclictest0-21swapper/321:07:073
536399200,0cyclictest0-21swapper/219:52:072
5357992019,0cyclictest0-21swapper/119:27:061
5353992020,0cyclictest10564-21apt-get19:22:060
5353992020,0cyclictest0-21swapper/020:50:000
5353992020,0cyclictest0-21swapper/020:32:070
535399200,0cyclictest0-21swapper/020:02:070
535399200,0cyclictest0-21swapper/019:12:050
5367991919,0cyclictest22182-21apt-get19:45:023
5363991919,0cyclictest0-21swapper/219:37:072
5363991917,1cyclictest606-21dbus-daemon20:35:002
5363991917,1cyclictest28664-21munin-run21:05:012
5357991919,0cyclictest0-21swapper/120:32:071
5357991919,0cyclictest0-21swapper/120:22:071
5367991818,0cyclictest7418-21apt-get20:22:073
5367991818,0cyclictest0-21swapper/320:37:053
5367991818,0cyclictest0-21swapper/319:32:063
5367991817,1cyclictest32208-21munin-run20:05:003
5357991816,1cyclictest11772-21turbostat20:30:011
535399180,0cyclictest0-21swapper/020:12:050
5367991717,0cyclictest0-21swapper/320:12:053
5363991717,0cyclictest0-21swapper/219:45:022
5363991716,1cyclictest15408-21apt-get19:32:072
5357991717,0cyclictest0-21swapper/119:37:071
5357991716,0cyclictest0-21swapper/119:22:071
5353991715,1cyclictest10137-21perf19:20:000
535399170,2cyclictest2189-21munin-run20:10:010
535399170,16cyclictest11811-21latency_hist20:30:010
5367991616,0cyclictest0-21swapper/320:47:083
5367991615,1cyclictest0-21swapper/319:27:063
5367991615,0cyclictest12212-21apt-get20:32:073
5367991614,1cyclictest25323-21taskset20:57:123
5363991616,0cyclictest0-21swapper/220:22:072
5363991616,0cyclictest0-21swapper/219:57:082
5363991615,1cyclictest17075-21apt-get20:42:072
5363991614,1cyclictest4589-21munin-run20:15:012
5363991613,2cyclictest23859-21perf20:55:002
536399160,1cyclictest0-21swapper/219:15:012
536399160,15cyclictest0-21swapper/220:57:112
5357991615,0cyclictest0-21swapper/120:42:071
5357991615,0cyclictest0-21swapper/120:12:051
5353991616,0cyclictest0-21swapper/020:42:070
5353991616,0cyclictest0-21swapper/019:52:080
5353991616,0cyclictest0-21swapper/019:47:070
5353991616,0cyclictest0-21swapper/019:32:070
5353991614,1cyclictest0-21swapper/019:30:010
535399161,14cyclictest26645-21/usr/sbin/munin21:02:150
535799150,1cyclictest0-21swapper/120:57:121
535399150,14cyclictest0-21swapper/021:07:140
536799140,14cyclictest0-21swapper/320:43:033
536799140,14cyclictest0-21swapper/319:35:013
536799140,0cyclictest0-21swapper/320:15:443
536399140,14cyclictest0-21swapper/220:04:522
5357991414,0cyclictest0-21swapper/119:15:161
535799140,14cyclictest0-21swapper/121:02:151
535799140,0cyclictest0-21swapper/119:55:071
535399140,0cyclictest0-21swapper/020:37:050
5363991313,0cyclictest0-21swapper/220:26:442
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional