You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 15:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Mon May 11, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
286422180104,28sleep10-21swapper/107:08:511
284272178106,28sleep20-21swapper/207:06:022
285822163113,28sleep30-21swapper/307:08:053
284772160103,27sleep00-21swapper/007:06:400
227452790,2sleep20-21swapper/209:14:462
262932760,6sleep126294-21cpuspeed_turbos12:05:131
198232550,2sleep20-21swapper/209:35:292
268742540,1sleep30-21swapper/310:53:243
268742540,1sleep30-21swapper/310:53:233
137862540,8sleep334-21ksoftirqd/309:31:093
308382490,1sleep10-21swapper/112:31:451
29010994722,2cyclictest0-21swapper/309:23:243
23792470,1sleep10-21swapper/111:23:181
174492470,1sleep30-21swapper/310:45:413
239872430,2chrt0-21swapper/311:39:473
239872430,2chrt0-21swapper/311:39:473
29008994111,12cyclictest0-21swapper/211:50:112
143022410,2sleep00-21swapper/011:08:000
2900899399,15cyclictest0-21swapper/212:26:302
2900899388,9cyclictest15303-21ssh10:45:032
2900899384,3cyclictest28170-21ssh11:42:132
29008993830,5cyclictest17807-21cut09:35:002
29008993812,16cyclictest28723-21sh09:42:332
56062370,1sleep10-21swapper/109:25:201
2900899378,18cyclictest15535-1kworker/u9:112:11:502
2900899369,9cyclictest24528-21chrt10:51:022
2900899369,9cyclictest24528-21chrt10:51:022
2900899369,18cyclictest26179-21kworker/u8:011:20:122
2900899367,13cyclictest0-21swapper/210:20:192
2900899367,12cyclictest11395-21rm09:54:072
2900899366,20cyclictest0-21swapper/210:41:342
29008993610,9cyclictest4024-21ssh11:48:482
29008993610,8cyclictest10137-21runrttasks11:29:202
29008993610,11cyclictest20187-21ssh11:35:562
29008993610,11cyclictest20187-21ssh11:35:562
2900899358,8cyclictest22066-21ssh10:01:172
2900899358,16cyclictest22788-21ssh10:25:302
2900899357,10cyclictest26939-21sh10:05:152
29008993511,9cyclictest13578-21latency09:55:182
2900899349,9cyclictest29692-21rm11:19:582
2900899348,9cyclictest9520-21ssh09:28:392
2900899348,9cyclictest9206-21ssh12:15:442
2900899348,8cyclictest9470-21ssh10:15:382
2900899348,8cyclictest15727-21ssh11:09:272
2900899348,16cyclictest16173-21diskmemload10:30:512
2900899345,12cyclictest16173-21diskmemload12:01:422
29008993428,5cyclictest0-21swapper/209:23:232
29008993411,9cyclictest28373-21sh12:05:552
29008993411,9cyclictest16038-21ssh11:57:122
29008993411,8cyclictest16173-21diskmemload11:02:262
2900599341,30cyclictest6616-21hddtemp_smartct09:50:180
2900899338,9cyclictest6757-21ssh10:38:262
2900899338,8cyclictest0-21swapper/210:14:222
2900899338,11cyclictest3618-21df_inode07:25:132
2900899338,10cyclictest14249-21ssh11:31:432
2900899337,9cyclictest30663-21ssh12:31:352
2900899337,9cyclictest2317-21rm09:46:472
2900899337,13cyclictest30964-21kworker/u8:111:10:082
29008993310,9cyclictest31155-21sh10:55:572
2900699331,29cyclictest30781-21ssh10:08:031
2900899329,8cyclictest2213-21cpuspeed_turbos12:35:142
2900899328,9cyclictest11426-21awk08:55:262
2900899322,28cyclictest0-21swapper/208:46:312
2900699321,7cyclictest15291-21ssh09:32:431
2900699321,2cyclictest141rcu_preempt11:14:391
29005993221,6cyclictest13803-21ssh09:31:100
128102320,2sleep30-21swapper/311:06:323
29010993123,6cyclictest6939-21ssh09:26:033
2900899319,8cyclictest20118-21users08:00:262
2900899318,14cyclictest0-21swapper/209:16:332
2900899316,9cyclictest30174-21mailstats07:10:232
2901099301,5cyclictest16173-21diskmemload12:11:193
2900899308,8cyclictest16246-21ssh12:21:012
2900899307,7cyclictest32378-21ntpq08:30:202
2900899306,8cyclictest5399-21/usr/sbin/munin07:30:212
2900899306,8cyclictest0-21swapper/208:40:132
2900699301,27cyclictest4072-21sh12:12:181
2900599301,27cyclictest16173-21diskmemload10:13:400
2900899299,8cyclictest2899-21chrt08:37:472
2900899298,8cyclictest0-21swapper/207:35:172
2900899296,8cyclictest0-21swapper/208:50:142
29006992924,3cyclictest16336-21ssh09:57:181
29006992923,4cyclictest5293-21ssh10:13:121
29006992916,10cyclictest14053-21cron07:50:001
2900699291,3cyclictest52950irq/141-enp3s0-09:40:461
2900699291,26cyclictest13726-21ssh10:43:431
29005992923,3cyclictest2702-21users08:35:250
2900599291,4cyclictest8722-21ssh09:27:500
29010992822,4cyclictest53050irq/142-enp3s0-09:35:233
2901099281,3cyclictest5562-21ssh11:01:013
2901099281,25cyclictest31618-21ssh10:32:363
2901099281,25cyclictest22208-21ssh09:14:153
2900899285,8cyclictest0-21swapper/207:15:242
29008992812,13cyclictest25434-21fschecks_time08:15:142
2900699281,25cyclictest12356-21ssh10:18:341
2900699281,25cyclictest0-21swapper/111:17:181
2900699281,25cyclictest0-21swapper/110:27:011
2900699281,25cyclictest0-21swapper/109:35:221
2900599281,3cyclictest31033-21ssh10:08:160
2900599281,25cyclictest0-21swapper/011:00:280
2900599281,25cyclictest0-21swapper/008:51:530
78592270,2sleep00-21swapper/011:27:030
29010992723,2cyclictest16173-21diskmemload09:52:203
29010992722,3cyclictest4661-1kworker/u9:010:14:223
2901099271,3cyclictest4864-21users08:40:263
2901099271,24cyclictest16173-21diskmemload09:40:483
2901099271,24cyclictest0-21swapper/310:58:573
2901099270,25cyclictest19113-21ssh10:23:333
2900899278,12cyclictest0-21swapper/208:20:122
29008992713,11cyclictest30319-21head08:25:222
29006992724,2cyclictest0-21swapper/109:22:341
29006992723,2cyclictest0-21swapper/110:36:341
29006992723,2cyclictest0-21swapper/108:50:531
29006992720,5cyclictest20647-21latency_hist10:25:001
2900699271,2cyclictest0-21swapper/109:52:061
2900699271,25cyclictest0-21swapper/109:47:211
2900699271,25cyclictest0-21swapper/108:47:561
2900699271,24cyclictest0-21swapper/111:00:021
2900699271,24cyclictest0-21swapper/108:56:521
2900699271,24cyclictest0-21swapper/108:42:581
2900699271,24cyclictest0-21swapper/108:31:591
2900699270,26cyclictest2124-21ssh10:59:201
29005992721,4cyclictest20109-21ssh09:35:460
2900599271,2cyclictest0-21swapper/010:24:580
2900599271,25cyclictest0-21swapper/011:16:590
2900599271,25cyclictest0-21swapper/010:59:090
2900599271,25cyclictest0-21swapper/010:41:030
2900599271,25cyclictest0-21swapper/010:32:380
2900599271,25cyclictest0-21swapper/009:41:360
2900599271,24cyclictest0-21swapper/008:45:510
8112260,2sleep30-21swapper/311:21:453
43342260,3sleep00-21swapper/011:49:070
2901099261,2cyclictest0-21swapper/310:08:073
2901099261,24cyclictest0-21swapper/309:55:463
2901099261,24cyclictest0-21swapper/308:47:403
2901099261,23cyclictest0-21swapper/310:25:163
2901099261,23cyclictest0-21swapper/308:56:123
2901099261,23cyclictest0-21swapper/308:36:003
2901099261,23cyclictest0-21swapper/308:34:123
2901099260,24cyclictest16173-21diskmemload11:18:333
2900899269,5cyclictest12475-21diskstats09:00:132
2900899268,15cyclictest11955-21ls07:45:012
2900899267,5cyclictest0-21swapper/207:45:212
29008992610,13cyclictest0-21swapper/208:05:162
2900699261,24cyclictest0-21swapper/110:32:211
2900699261,24cyclictest0-21swapper/109:13:351
2900699260,24cyclictest0-21swapper/108:35:311
29005992624,1cyclictest0-21swapper/010:25:430
29005992624,1cyclictest0-21swapper/008:42:350
2900599261,24cyclictest0-21swapper/011:14:190
2900599261,24cyclictest0-21swapper/010:35:400
2900599261,24cyclictest0-21swapper/009:22:130
2900599261,24cyclictest0-21swapper/009:13:160
2900599261,1cyclictest0-21swapper/012:11:350
2900599261,1cyclictest0-21swapper/010:47:320
2900599261,1cyclictest0-21swapper/009:55:260
2900599261,1cyclictest0-21swapper/008:56:460
2901099251,1cyclictest0-21swapper/312:16:463
2901099251,1cyclictest0-21swapper/308:50:183
2901099250,2cyclictest11093-21ssh10:41:043
2901099250,23cyclictest0-21swapper/311:14:063
29008992515,7cyclictest15729-21tail07:50:252
29008992513,9cyclictest15280-21/usr/sbin/munin09:05:232
29008992510,5cyclictest16783-21df_inode07:55:152
2900699250,1cyclictest0-21swapper/112:16:571
29005992523,1cyclictest0-21swapper/010:15:060
287352254,15sleep329003-21cyclictest07:10:003
29010992422,1cyclictest0-21swapper/309:47:223
29008992415,7cyclictest2587-21grep07:20:262
29008992414,7cyclictest23432-21grep08:10:152
29006992416,6cyclictest13522-21ssh11:30:591
2900599249,3cyclictest66050lldpd11:50:280
288632240,2sleep30-21swapper/312:06:223
262642240,1sleep00-21swapper/009:16:330
126532240,2sleep00-21swapper/007:45:160
29010992321,1cyclictest0-21swapper/310:15:073
2900699233,14cyclictest636-21runrttasks10:00:121
29006992321,1cyclictest0-21swapper/110:45:461
29006992315,5cyclictest30929-21sh07:15:001
29005992321,1cyclictest0-21swapper/011:21:000
29005992321,1cyclictest0-21swapper/008:33:480
29005992316,5cyclictest22292-21ls12:25:240
284082230,9sleep334-21ksoftirqd/311:42:283
29010992214,5cyclictest26114-21grep08:15:213
29010992213,7cyclictest6229-21diskstats11:50:143
2901099221,12cyclictest34-21ksoftirqd/311:45:243
29005992212,7cyclictest23220-21df_inode08:10:140
2901099216,6cyclictest3281-21sendmail_mailqu10:35:233
29010992113,6cyclictest12934-21diskstats11:55:143
29010992112,7cyclictest14921-21netstat12:20:203
29010992110,10cyclictest32652-21ssh12:33:363
2900699218,2cyclictest141rcu_preempt11:35:551
2900699218,2cyclictest141rcu_preempt11:35:541
2900699213,2cyclictest25274-21ssh09:15:331
29006992113,5cyclictest28651-21latency_hist10:55:011
29006992113,5cyclictest28651-21latency_hist10:55:011
29006992110,3cyclictest22-21ksoftirqd/111:51:361
29005992119,1cyclictest0-21swapper/009:45:110
2900599211,7cyclictest12691-21hddtemp_smartct09:00:150
29005992112,7cyclictest29793-21date07:10:180
162072210,4sleep116204-21ls07:55:011
65762200,2chrt0-21swapper/307:30:223
29010992015,4cyclictest20107-21grep08:00:263
29010992012,6cyclictest15196-21ntp_kernel_err07:50:193
2900699208,8cyclictest30921-21cron11:45:001
2900699203,11cyclictest19296-21apt-get12:00:131
29006992014,4cyclictest5917-21timerandwakeup11:25:241
29006992014,3cyclictest858-21cron07:20:011
29006992012,6cyclictest1141-21seq11:45:541
2900699201,14cyclictest20700-21chrt12:25:041
29006992010,7cyclictest27795-21cat08:20:171
2900599209,9cyclictest12468-21ssh11:55:080
2900599208,3cyclictest141rcu_preempt10:05:000
2900599207,11cyclictest541-21ssh12:33:570
29005992018,1cyclictest0-21swapper/012:16:460
2900599201,5cyclictest0-21swapper/010:50:140
2900599201,5cyclictest0-21swapper/010:50:130
29005992012,6cyclictest9625-21date07:40:000
29005992012,6cyclictest30981-21seq08:27:060
29005992012,6cyclictest12155-21ls11:30:190
2901099198,6cyclictest53050irq/142-enp3s0-12:25:133
29010991914,3cyclictest11129-21sh11:30:073
29010991913,4cyclictest1810-21kernelversion07:20:183
29010991913,3cyclictest24769-21sh08:15:003
29010991912,5cyclictest4403-21mailstats07:25:223
29010991911,6cyclictest5423-21ls11:25:193
29010991911,6cyclictest25830-21basename10:05:003
29010991911,6cyclictest24430-21ntpq09:15:193
2900699199,7cyclictest8536-21cut12:40:011
2900699199,2cyclictest22-21ksoftirqd/107:40:141
2900699197,3cyclictest22-21ksoftirqd/111:55:131
2900699195,3cyclictest141rcu_preempt11:05:211
2900699194,2cyclictest22-21ksoftirqd/112:23:401
29006991913,4cyclictest636-21runrttasks08:00:161
29006991912,5cyclictest22-21ksoftirqd/107:40:001
29006991912,5cyclictest22-21ksoftirqd/107:24:591
29006991910,6cyclictest3393-21grep07:25:131
29006991910,6cyclictest23355-21seq08:10:161
2900599193,11cyclictest1142-21apt-get07:20:130
29005991912,5cyclictest26774-21seq08:18:230
29005991912,5cyclictest18236-21seq07:58:530
29005991911,6cyclictest31861-21perl07:15:150
29005991911,5cyclictest3091-21awk12:35:210
29005991910,6cyclictest19499-21timerwakeupswit11:35:260
29005991910,6cyclictest19499-21timerwakeupswit11:35:250
2901099189,6cyclictest29800-21fw_packets08:25:163
2901099182,11cyclictest27189-21apt-get08:20:123
29010991813,3cyclictest31262-21apt-config07:15:013
29010991812,4cyclictest21684-21sh12:01:063
29010991811,5cyclictest560-21dbus-daemon09:05:003
29010991811,5cyclictest17600-21mailstats07:55:233
29010991810,6cyclictest15183-21ntpq09:05:203
29010991810,5cyclictest53050irq/142-enp3s0-12:35:133
2901099180,7cyclictest21292-21gltestperf08:05:173
2900699188,7cyclictest26237-21cat08:15:211
2900699182,14cyclictest5939-21cut07:30:151
2900599186,2cyclictest25167-21perl11:40:160
2900599183,3cyclictest141rcu_preempt08:24:590
29005991814,3cyclictest12-21ksoftirqd/008:10:010
29005991813,3cyclictest6940-21vmstat07:30:250
29005991813,3cyclictest28041-21sh12:05:340
29005991813,3cyclictest16205-21sh07:55:000
29005991812,4cyclictest4115-21needreboot07:25:190
29005991811,5cyclictest19524-21expr12:00:140
2900599181,14cyclictest16173-21diskmemload12:21:250
29005991810,6cyclictest15742-21sed09:05:260
2901099178,7cyclictest560-21dbus-daemon07:44:593
29010991712,3cyclictest9551-21latency_hist07:40:003
29010991710,4cyclictest12938-21missed_timers07:45:203
2900699178,7cyclictest22-21ksoftirqd/107:55:161
2900699176,2cyclictest22-21ksoftirqd/109:09:591
29006991712,3cyclictest30419-21sendmail_mailqu08:25:221
29006991711,4cyclictest21295-21cut08:05:161
2900599179,6cyclictest10332-21grep07:40:160
2900599169,5cyclictest18679-21cat08:00:130
2900699151,3cyclictest12494-21fschecks_count09:00:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional