You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 20:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Tue Aug 25, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1887499260,25cyclictest0-21swapper/009:13:370
1887799250,24cyclictest8175-21sh09:34:263
1887799250,24cyclictest0-21swapper/310:13:343
18876992523,1cyclictest607-21in:imuxsock08:10:012
18875992523,1cyclictest21574-21cron10:15:001
1887499253,1cyclictest13-21ksoftirqd/009:54:540
1887499250,24cyclictest24061-21apt09:20:000
1887499250,24cyclictest12255-21diskmemload09:27:260
1887499250,23cyclictest313-21systemd-journal08:10:010
18877992423,0cyclictest12255-21diskmemload10:01:003
18877992422,1cyclictest34-21ksoftirqd/312:35:003
18877992422,1cyclictest0-21swapper/312:38:243
18877992421,1cyclictest151rcu_preempt12:10:013
1887799240,23cyclictest4695-21sendmail-msp10:00:003
1887799240,23cyclictest30596-21sh10:22:043
1887799240,23cyclictest12255-21diskmemload12:15:143
1887799240,23cyclictest12255-21diskmemload11:55:143
1887799240,23cyclictest12255-21diskmemload11:12:063
1887799240,23cyclictest0-21swapper/311:01:233
1887699240,24cyclictest32313-21ssh11:50:072
1887699240,24cyclictest0-21swapper/210:47:442
18875992421,2cyclictest25586-21cat09:50:011
1887599240,23cyclictest3372-21apt-get07:45:021
1887599240,23cyclictest13139-21ssh11:32:551
1887599240,1cyclictest12255-21diskmemload11:23:321
18874992422,2cyclictest0-21swapper/011:06:150
1887499240,24cyclictest0-21swapper/010:37:240
1887499240,23cyclictest27020-21ssh10:19:460
1887499240,23cyclictest22093-21munin-run12:10:010
1887499240,23cyclictest12255-21diskmemload10:47:320
1887499240,23cyclictest12255-21diskmemload09:22:300
1887499240,23cyclictest0-21swapper/010:56:450
1887499240,1cyclictest0-21swapper/011:03:540
1887499240,0cyclictest0-21swapper/012:02:420
1887499240,0cyclictest0-21swapper/010:44:450
18877992322,0cyclictest34-21ksoftirqd/309:13:353
18877992322,0cyclictest12255-21diskmemload11:54:563
18877992321,1cyclictest13131-21sh11:32:553
18877992321,1cyclictest0-21swapper/312:05:483
18877992321,1cyclictest0-21swapper/310:56:253
18877992321,1cyclictest0-21swapper/309:41:343
18877992321,1cyclictest0-21swapper/308:50:013
18877992321,0cyclictest151rcu_preempt10:28:463
18877992320,2cyclictest24757-21ssh10:45:443
1887799230,23cyclictest0-21swapper/311:38:183
1887799230,23cyclictest0-21swapper/311:07:273
1887799230,22cyclictest8720-21sh11:29:333
1887799230,22cyclictest26377-21apt-get07:25:373
1887799230,22cyclictest21660-21sh09:45:453
1887799230,22cyclictest16235-21ssh12:04:383
1887799230,22cyclictest12255-21diskmemload09:29:233
1887799230,22cyclictest11468-21ssh12:29:003
1887799230,22cyclictest0-21swapper/311:21:273
1887799230,0cyclictest22325-21apt-get10:15:383
18876992323,0cyclictest12255-21diskmemload09:30:252
18876992322,1cyclictest12255-21diskmemload09:51:252
18876992320,2cyclictest32573-21expr09:55:382
1887699230,2cyclictest19233-21rm12:06:012
1887699230,23cyclictest12255-21diskmemload12:18:402
1887699230,23cyclictest12255-21diskmemload10:38:112
1887699230,23cyclictest0-21swapper/212:36:512
1887699230,23cyclictest0-21swapper/210:33:402
1887699230,23cyclictest0-21swapper/209:25:162
1887699230,22cyclictest29056-21ssh10:50:292
1887699230,22cyclictest24854-21ssh11:15:072
1887699230,22cyclictest16117-21ssh12:32:052
1887699230,22cyclictest13321-21munin-run11:05:002
1887699230,22cyclictest0-21swapper/209:12:182
1887699230,0cyclictest0-21swapper/212:27:362
1887699230,0cyclictest0-21swapper/210:07:192
1887699230,0cyclictest0-21swapper/210:04:432
18875992323,0cyclictest12255-21diskmemload12:05:061
18875992323,0cyclictest0-21swapper/111:29:391
18875992323,0cyclictest0-21swapper/110:07:061
18875992322,1cyclictest5350-21/usr/sbin/munin09:30:471
18875992322,1cyclictest17563-21ssh10:40:141
18875992321,1cyclictest32009-21ssh12:18:111
18875992321,1cyclictest2682-21ssh09:57:211
18875992321,1cyclictest21626-21sh12:36:511
18875992320,2cyclictest30418-21sh10:21:421
1887599230,23cyclictest12255-21diskmemload09:43:041
1887599230,23cyclictest0-21swapper/112:27:301
1887599230,23cyclictest0-21swapper/112:10:461
1887599230,23cyclictest0-21swapper/110:30:321
1887599230,23cyclictest0-21swapper/110:15:381
1887599230,22cyclictest8203-21ssh11:00:161
1887599230,22cyclictest21939-21sh11:11:171
1887599230,22cyclictest2108-21ssh10:55:181
1887599230,22cyclictest15945-21ssh11:05:511
1887599230,22cyclictest12255-21diskmemload10:36:581
1887599230,22cyclictest11471-21sh09:36:121
1887599230,0cyclictest12255-21diskmemload11:50:161
1887599230,0cyclictest12255-21diskmemload11:15:431
1887599230,0cyclictest0-21swapper/112:22:531
1887599230,0cyclictest0-21swapper/109:18:481
18874992323,0cyclictest12255-21diskmemload11:46:340
18874992323,0cyclictest0-21swapper/012:26:550
18874992323,0cyclictest0-21swapper/008:50:370
18874992322,1cyclictest62550irq/127-eno110:12:520
1887499231,21cyclictest62550irq/127-eno111:55:380
1887499231,21cyclictest62550irq/127-eno110:28:150
1887499230,23cyclictest12255-21diskmemload10:22:530
1887499230,23cyclictest0-21swapper/011:12:010
1887499230,23cyclictest0-21swapper/010:51:280
1887499230,22cyclictest6218-21sh09:31:540
1887499230,22cyclictest5090-21sh10:00:140
1887499230,22cyclictest28484-21awk07:30:000
1887499230,22cyclictest24251-21sh11:42:340
1887499230,22cyclictest15008-21diskmemload11:35:060
1887499230,22cyclictest0-21swapper/012:13:520
1887499230,22cyclictest0-21swapper/009:58:540
1887499230,22cyclictest0-21swapper/009:45:470
1887499230,0cyclictest12255-21diskmemload12:31:340
1887499230,0cyclictest0-21swapper/012:17:040
18877992222,0cyclictest0-21swapper/310:50:573
18877992222,0cyclictest0-21swapper/310:33:113
18877992220,1cyclictest0-21swapper/312:25:003
18877992220,1cyclictest0-21swapper/310:35:063
18877992220,1cyclictest0-21swapper/310:06:233
1887799220,21cyclictest8914-21apt-get07:55:363
1887799220,21cyclictest32646-21perf08:45:013
1887799220,21cyclictest32646-21perf08:45:013
1887799220,21cyclictest17782-21ssh10:40:303
1887799220,21cyclictest12255-21diskmemload09:35:103
1887799220,21cyclictest0-21swapper/311:15:183
1887799220,21cyclictest0-21swapper/309:53:383
1887799220,21cyclictest0-21swapper/308:10:373
1887799220,1cyclictest0-21swapper/309:20:293
1887799220,0cyclictest12255-21diskmemload11:45:573
18876992222,0cyclictest12255-21diskmemload11:27:182
18876992222,0cyclictest0-21swapper/209:41:392
18876992222,0cyclictest0-21swapper/209:20:212
18876992221,1cyclictest2361-21apt-get10:55:392
18876992220,1cyclictest5876-21apt-get11:55:382
18876992220,1cyclictest24350-21ssh10:16:092
18876992220,1cyclictest12255-21diskmemload10:25:292
1887699221,21cyclictest19183-21rm11:10:012
1887699220,2cyclictest16954-21ssh11:07:162
1887699220,22cyclictest0-21swapper/211:48:532
1887699220,21cyclictest31025-21ssh10:22:362
1887699220,21cyclictest2763-21ssh12:20:432
1887699220,21cyclictest11533-21apt-get12:00:372
1887699220,21cyclictest0-21swapper/211:20:472
1887699220,21cyclictest0-21swapper/210:40:302
1887699220,1cyclictest12536-21ssh09:37:372
1887699220,1cyclictest0-21swapper/208:25:002
18875992222,0cyclictest12255-21diskmemload11:35:161
18875992222,0cyclictest0-21swapper/112:34:451
18875992222,0cyclictest0-21swapper/112:00:121
18875992222,0cyclictest0-21swapper/111:45:051
18875992222,0cyclictest0-21swapper/109:23:271
18875992221,1cyclictest639-21ssh10:25:101
18875992220,1cyclictest0-21swapper/107:30:001
1887599220,22cyclictest0-21swapper/110:00:411
1887599220,22cyclictest0-21swapper/109:50:111
1887599220,21cyclictest9878-21perf09:05:001
1887599220,21cyclictest8914-21apt-get07:55:371
1887599220,21cyclictest6306-21ssh11:55:391
1887599220,21cyclictest23555-21ssh10:45:371
1887599220,1cyclictest23076-21ssh11:40:581
1887599220,1cyclictest1278-21ssh09:27:491
1887599220,1cyclictest12248-21cron09:10:001
1887599220,1cyclictest0-21swapper/110:52:041
1887599220,1cyclictest0-21swapper/109:10:391
18874992221,1cyclictest0-21swapper/011:52:480
18874992220,1cyclictest12255-21diskmemload12:20:380
18874992220,1cyclictest0-21swapper/008:30:000
1887499221,0cyclictest13-21ksoftirqd/010:30:250
1887499220,22cyclictest0-21swapper/009:35:250
1887499220,21cyclictest9425-21ssh11:30:070
1887499220,21cyclictest12255-21diskmemload09:42:100
1887499220,21cyclictest0-21swapper/012:35:010
1887499220,21cyclictest0-21swapper/011:27:160
1887499220,21cyclictest0-21swapper/011:21:290
1887499220,21cyclictest0-21swapper/008:15:000
1887499220,1cyclictest606-21dbus-daemon07:20:010
1887499220,0cyclictest0-21swapper/011:15:200
18877992120,0cyclictest0-21swapper/308:20:373
1887799210,1cyclictest13439-21perf08:05:013
1887799210,1cyclictest0-21swapper/307:15:393
18876992119,2cyclictest20849-21ssh11:40:172
18876992119,1cyclictest25441-21munin-run08:30:002
1887699210,21cyclictest0-21swapper/212:10:172
1887699210,20cyclictest7790-21apt-get09:00:372
1887699210,20cyclictest30546-21apt-get08:40:362
1887699210,20cyclictest30546-21apt-get08:40:362
1887699210,20cyclictest16303-21ssh10:10:132
1887699210,20cyclictest0-21swapper/208:50:372
1887699210,0cyclictest23983-21apt-get07:20:372
1887699210,0cyclictest0-21swapper/211:36:342
18875992120,1cyclictest18514-21apt-get08:15:361
1887599210,0cyclictest23334-21apt-get08:25:351
1887599210,0cyclictest0-21swapper/108:45:361
1887499210,21cyclictest0-21swapper/009:05:380
1887499210,0cyclictest0-21swapper/007:20:370
1887799200,20cyclictest0-21swapper/307:20:373
1887799200,0cyclictest0-21swapper/308:05:353
18876992020,0cyclictest0-21swapper/208:00:372
18876992020,0cyclictest0-21swapper/207:15:392
18876992019,1cyclictest18212-21perf08:15:002
18876992018,1cyclictest28481-21munin-run07:30:002
18875992020,0cyclictest2923-21apt-get08:50:371
18875992020,0cyclictest21515-21apt-get07:15:391
18874992020,0cyclictest0-21swapper/008:45:360
18874992019,1cyclictest4125-21apt-get07:45:380
18874992018,1cyclictest11030-21grep08:00:010
1887499200,20cyclictest0-21swapper/007:40:380
18877991919,0cyclictest28775-21apt-get07:30:363
18877991919,0cyclictest18514-21apt-get08:15:373
18877991919,0cyclictest0-21swapper/307:10:363
18877991917,1cyclictest0-21swapper/307:50:383
18876991919,0cyclictest0-21swapper/209:05:382
18876991919,0cyclictest0-21swapper/207:50:382
18876991919,0cyclictest0-21swapper/207:40:382
18875991919,0cyclictest19118-21apt-get07:10:361
18875991919,0cyclictest0-21swapper/108:05:351
18875991919,0cyclictest0-21swapper/108:00:371
18875991919,0cyclictest0-21swapper/107:35:361
18875991919,0cyclictest0-21swapper/107:20:371
18874991919,0cyclictest0-21swapper/008:00:370
18874991918,0cyclictest0-21swapper/008:15:370
1887499190,18cyclictest31170-21apt-get07:35:360
18877991818,0cyclictest0-21swapper/309:00:373
18877991816,1cyclictest7495-21awk09:00:013
18877991816,1cyclictest34-21ksoftirqd/307:40:003
1887699181,16cyclictest0-21swapper/208:45:362
18875991816,1cyclictest30875-21perf07:35:001
18874991817,0cyclictest0-21swapper/008:40:360
18874991817,0cyclictest0-21swapper/008:40:360
18877991716,0cyclictest28133-21apt-get08:35:393
18877991715,2cyclictest25733-21apt-get08:30:373
18876991717,0cyclictest0-21swapper/208:35:392
18876991717,0cyclictest0-21swapper/207:30:372
18876991717,0cyclictest0-21swapper/207:10:362
18875991717,0cyclictest20939-21apt-get08:20:371
18875991715,1cyclictest3387-21munin-run07:45:001
18874991717,0cyclictest0-21swapper/007:10:360
18874991716,1cyclictest0-21swapper/008:55:390
18876991616,0cyclictest0-21swapper/207:45:382
18876991616,0cyclictest0-21swapper/207:35:362
18876991615,1cyclictest11026-21perf08:00:012
18876991615,0cyclictest0-21swapper/208:15:372
18875991614,1cyclictest0-21swapper/109:00:011
18874991616,0cyclictest0-21swapper/007:50:380
1887499160,0cyclictest0-21swapper/008:35:390
18877991514,1cyclictest0-21swapper/307:40:383
18875991515,0cyclictest6520-21apt-get07:50:381
18875991515,0cyclictest0-21swapper/108:30:371
18874991515,0cyclictest0-21swapper/009:00:370
1887799140,14cyclictest0-21swapper/309:05:263
1887799140,14cyclictest0-21swapper/308:50:153
1887799140,14cyclictest0-21swapper/308:26:183
1887799140,14cyclictest0-21swapper/307:45:183
1887699140,14cyclictest0-21swapper/208:55:152
1887699140,14cyclictest0-21swapper/208:30:302
1887599142,5cyclictest0-21swapper/108:10:371
1887599140,14cyclictest0-21swapper/108:35:081
1887599140,0cyclictest0-21swapper/108:40:041
1887599140,0cyclictest0-21swapper/108:40:041
18874991414,0cyclictest0-21swapper/008:30:170
18874991414,0cyclictest0-21swapper/008:20:020
18874991414,0cyclictest0-21swapper/007:30:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional