You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 23:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Tue Aug 25, 2026 12:45:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18282184163,14sleep30-21swapper/307:05:093
37122171106,29sleep10-21swapper/107:09:471
36682170101,28sleep20-21swapper/207:09:122
34012166103,27sleep00-21swapper/007:05:460
315882690,1sleep10-21swapper/112:28:461
294482650,9sleep122-21ksoftirqd/111:39:011
321152630,10sleep334-21ksoftirqd/310:28:293
4932620,1sleep20-21swapper/211:40:462
138522620,1sleep10-21swapper/111:27:021
138522620,1sleep10-21swapper/111:27:011
178422570,1sleep00-21swapper/011:54:460
71562550,1sleep20-21swapper/212:35:012
248392550,2sleep30-21swapper/310:46:473
248392550,2sleep30-21swapper/310:46:473
326632540,1sleep20-21swapper/209:40:202
306192540,1sleep20-21swapper/210:03:072
306192540,1sleep20-21swapper/210:03:072
261702530,1sleep20-21swapper/211:35:422
245502520,2sleep20-21swapper/209:10:172
72032470,2sleep00-21swapper/010:34:150
294592470,1sleep20-21swapper/212:02:592
107732450,4sleep334-21ksoftirqd/307:25:133
400399384,15cyclictest6041-21ssh12:10:000
4004993710,10cyclictest22-21ksoftirqd/110:34:591
400399378,12cyclictest5758-21basename09:45:000
400399375,20cyclictest0-21swapper/009:55:110
4004993511,12cyclictest22-21ksoftirqd/109:45:011
400399358,9cyclictest16068-21ssh10:40:200
400399358,16cyclictest23366-21diskmemload12:20:070
400399357,9cyclictest29856-21ssh09:14:480
400399357,10cyclictest23366-21diskmemload11:35:160
400399355,10cyclictest26467-21sh12:25:090
400399347,10cyclictest23366-21diskmemload12:33:460
400399345,9cyclictest1-21systemd10:14:000
400399344,11cyclictest23366-21diskmemload10:55:390
400399343,17cyclictest1099-21grep10:05:170
400399336,9cyclictest22387-21ssh11:33:380
400399336,11cyclictest10109-21idleruntime-cro11:25:000
400399335,12cyclictest9780-21ssh12:35:580
400399334,11cyclictest3969-21sh09:19:360
400399332,12cyclictest14513-21latency_hist07:35:010
4004993228,2cyclictest22-21ksoftirqd/108:26:211
400399327,10cyclictest23366-21diskmemload11:03:350
400399326,9cyclictest4952-21tr09:20:140
400399326,9cyclictest19567-21fschecks_count07:45:160
400399326,9cyclictest19567-21fschecks_count07:45:160
400399325,8cyclictest26899-21sh10:25:000
400399324,10cyclictest27738-21sed08:05:010
4003993216,11cyclictest16099-21ssh11:05:110
400399315,16cyclictest23366-21diskmemload10:03:360
400399315,16cyclictest23366-21diskmemload10:03:350
400399314,11cyclictest1876-21ssh11:42:080
400399313,8cyclictest24850-21sed09:35:000
4004993016,5cyclictest22-21ksoftirqd/110:40:271
400499300,6cyclictest560-21dbus-daemon11:30:031
400399307,10cyclictest23366-21diskmemload11:27:340
400399307,10cyclictest23366-21diskmemload11:27:330
400399306,9cyclictest31145-21grep11:15:250
400399306,9cyclictest23366-21diskmemload10:26:230
400399306,9cyclictest23366-21diskmemload10:15:330
400399306,14cyclictest28189-21rm12:01:420
400399305,9cyclictest6950-21missed_timers07:15:200
400399305,8cyclictest17457-21ssh09:29:270
400399305,12cyclictest20066-21timerwakeupswit11:55:240
400399304,6cyclictest0-21swapper/009:50:130
400399304,16cyclictest9511-21ssh12:11:510
400399304,16cyclictest9511-21ssh12:11:500
400399303,9cyclictest12494-21latency_hist08:45:000
158352300,1sleep00-21swapper/007:35:210
400699291,2cyclictest0-21swapper/309:00:233
400699291,25cyclictest9110-21memory07:20:193
400499295,10cyclictest22-21ksoftirqd/111:10:001
4004992915,5cyclictest22-21ksoftirqd/109:30:011
400399295,9cyclictest7496-21ssh11:46:020
400399295,9cyclictest32333-21tr08:15:130
400399295,8cyclictest27848-21sh10:49:490
400399295,8cyclictest27848-21sh10:49:480
400399295,11cyclictest23370-21latency_hist09:09:590
83712280,2chrt0-21swapper/208:35:112
83712280,2chrt0-21swapper/208:35:102
4006992821,4cyclictest24388-21open_inodes07:55:223
400499287,9cyclictest22-21ksoftirqd/110:14:021
400499281,25cyclictest0-21swapper/108:21:051
400399285,8cyclictest27216-21ssh09:35:480
400399285,14cyclictest25021-21seq07:56:160
400399284,9cyclictest4531-21tail07:10:160
400399284,9cyclictest23366-21diskmemload09:46:230
400399284,8cyclictest10464-21/usr/sbin/munin07:25:170
400399284,6cyclictest0-21swapper/010:36:560
400399283,10cyclictest23366-21diskmemload10:51:200
400399283,10cyclictest23265-21tune2fs11:10:150
400399283,10cyclictest20773-21awk09:00:260
400399282,11cyclictest30058-21/usr/sbin/munin08:10:200
400399281,25cyclictest0-21swapper/008:32:080
400699271,3cyclictest32561-21diskstats08:15:153
400699271,24cyclictest0-21swapper/310:30:373
400699271,24cyclictest0-21swapper/308:13:003
400699271,24cyclictest0-21swapper/307:35:283
400599271,2cyclictest0-21swapper/208:13:222
400599271,2cyclictest0-21swapper/207:36:142
400599271,25cyclictest28-21ksoftirqd/209:01:262
400499277,9cyclictest22-21ksoftirqd/110:26:411
400499277,11cyclictest22-21ksoftirqd/111:15:251
400499275,11cyclictest22-21ksoftirqd/109:50:141
400499275,10cyclictest22-21ksoftirqd/109:20:551
400499274,10cyclictest22-21ksoftirqd/112:05:221
4004992724,2cyclictest0-21swapper/108:30:561
4004992721,4cyclictest20126-21/usr/sbin/munin09:00:201
400499271,2cyclictest0-21swapper/108:03:441
400499271,25cyclictest0-21swapper/107:36:371
400499271,24cyclictest0-21swapper/109:05:441
400499271,24cyclictest0-21swapper/108:50:051
400499271,24cyclictest0-21swapper/108:48:221
400499271,24cyclictest0-21swapper/108:15:491
400499271,24cyclictest0-21swapper/107:50:331
400499271,24cyclictest0-21swapper/107:46:071
400499271,24cyclictest0-21swapper/107:46:061
400499271,24cyclictest0-21swapper/107:42:311
400499271,24cyclictest0-21swapper/107:25:491
4004992710,5cyclictest22-21ksoftirqd/109:15:511
400399272,8cyclictest0-21swapper/008:24:210
400399271,24cyclictest0-21swapper/008:50:020
400399271,24cyclictest0-21swapper/008:06:210
400399271,24cyclictest0-21swapper/007:50:260
400399271,16cyclictest0-21swapper/012:19:320
400399270,25cyclictest12-21ksoftirqd/007:43:030
260732270,1sleep326074-21/usr/sbin/munin09:35:193
400699261,2cyclictest0-21swapper/308:39:363
400699261,2cyclictest0-21swapper/308:39:353
400699261,2cyclictest0-21swapper/307:45:023
400699261,2cyclictest0-21swapper/307:45:023
400699261,2cyclictest0-21swapper/307:42:443
400699261,23cyclictest0-21swapper/308:55:523
400699261,23cyclictest0-21swapper/308:49:223
400699261,23cyclictest0-21swapper/308:26:353
400699261,23cyclictest0-21swapper/308:05:023
400699261,23cyclictest0-21swapper/307:51:483
400699261,18cyclictest0-21swapper/308:00:423
400699260,24cyclictest0-21swapper/309:05:553
4005992623,2cyclictest0-21swapper/207:26:352
400599261,2cyclictest0-21swapper/208:20:082
400599261,2cyclictest0-21swapper/208:01:382
400599261,24cyclictest0-21swapper/208:25:482
400599261,23cyclictest0-21swapper/209:06:272
400599261,23cyclictest0-21swapper/208:48:222
400599261,23cyclictest0-21swapper/208:30:062
400599261,23cyclictest0-21swapper/208:05:032
400599261,23cyclictest0-21swapper/207:50:262
400599261,23cyclictest0-21swapper/207:45:022
400599261,23cyclictest0-21swapper/207:45:012
400599261,23cyclictest0-21swapper/207:32:272
400599261,23cyclictest0-21swapper/207:22:342
4005992610,4cyclictest66050lldpd09:19:232
400599260,24cyclictest0-21swapper/208:51:002
400499269,8cyclictest22-21ksoftirqd/111:54:061
400499268,2cyclictest141rcu_preempt09:38:101
400499267,10cyclictest22-21ksoftirqd/110:17:201
400499263,5cyclictest22-21ksoftirqd/112:20:261
4004992624,1cyclictest0-21swapper/108:10:071
400499261,6cyclictest0-21swapper/111:45:131
4004992613,5cyclictest22-21ksoftirqd/109:32:061
400499261,24cyclictest0-21swapper/107:57:171
400499261,24cyclictest0-21swapper/107:30:001
400499261,1cyclictest0-21swapper/108:57:581
400499261,1cyclictest0-21swapper/108:05:041
4004992611,2cyclictest141rcu_preempt12:00:241
4004992610,4cyclictest22-21ksoftirqd/109:57:141
400499260,7cyclictest32311-21ls11:40:251
400499260,24cyclictest0-21swapper/107:20:411
400499260,17cyclictest22-21ksoftirqd/111:55:181
4003992624,1cyclictest0-21swapper/008:57:080
400399261,24cyclictest0-21swapper/007:20:490
400399261,1cyclictest0-21swapper/008:39:590
400399261,1cyclictest0-21swapper/008:39:580
400399260,24cyclictest0-21swapper/008:25:020
233082260,2sleep30-21swapper/310:21:213
188452260,9sleep228-21ksoftirqd/210:18:412
160502260,2sleep30-21swapper/312:16:393
12922260,2chrt0-21swapper/210:29:592
400699251,2cyclictest0-21swapper/308:51:443
400699251,23cyclictest0-21swapper/308:21:163
400699250,23cyclictest0-21swapper/308:30:083
400699250,23cyclictest0-21swapper/307:32:153
4005992523,1cyclictest0-21swapper/207:56:192
400599251,2cyclictest0-21swapper/208:55:532
400599251,23cyclictest0-21swapper/208:44:212
400599251,23cyclictest0-21swapper/207:42:192
400599251,1cyclictest0-21swapper/208:15:012
400499259,4cyclictest22-21ksoftirqd/110:09:201
400499258,5cyclictest22-21ksoftirqd/110:23:401
400499258,4cyclictest22-21ksoftirqd/112:37:221
400499255,10cyclictest22-21ksoftirqd/111:23:421
400499254,6cyclictest22-21ksoftirqd/111:04:011
4004992512,5cyclictest23865-21ntp_states11:10:221
400499251,1cyclictest0-21swapper/108:41:071
4004992510,9cyclictest22-21ksoftirqd/110:48:131
4004992510,9cyclictest22-21ksoftirqd/110:48:121
400499250,6cyclictest12406-21ssh09:49:591
4003992523,1cyclictest0-21swapper/008:49:170
279322250,2sleep20-21swapper/210:49:552
279322250,2sleep20-21swapper/210:49:542
269542250,2sleep3331rcuc/312:00:263
4006992422,1cyclictest0-21swapper/308:43:013
400599240,12cyclictest210782ssh10:44:432
400499247,9cyclictest22-21ksoftirqd/110:39:041
400499245,11cyclictest22-21ksoftirqd/109:10:381
400499241,13cyclictest22-21ksoftirqd/112:30:251
57082231,7sleep228-21ksoftirqd/210:32:462
400699234,3cyclictest141rcu_preempt12:20:203
4006992314,6cyclictest52750irq/139-enp3s0-12:05:043
400599234,3cyclictest23366-21diskmemload11:19:442
400599231,5cyclictest12952-21ps09:25:232
400499238,3cyclictest141rcu_preempt10:55:261
400499237,3cyclictest141rcu_preempt12:10:101
400499237,3cyclictest141rcu_preempt12:10:101
400499231,8cyclictest22-21ksoftirqd/112:15:191
400499231,4cyclictest22-21ksoftirqd/110:50:251
4006992211,6cyclictest52750irq/139-enp3s0-11:45:113
400599228,3cyclictest52552rm10:08:282
400599224,3cyclictest23366-21diskmemload12:05:532
400599224,3cyclictest23366-21diskmemload11:30:512
400599224,3cyclictest23366-21diskmemload10:36:232
400599224,3cyclictest23366-21diskmemload09:55:092
400599223,4cyclictest23366-21diskmemload11:55:032
400599223,3cyclictest23366-21diskmemload12:27:412
400599223,3cyclictest23366-21diskmemload10:53:002
400599223,3cyclictest23366-21diskmemload09:45:192
400599222,4cyclictest23366-21diskmemload11:02:142
400599222,3cyclictest23366-21diskmemload11:54:252
400599222,3cyclictest23366-21diskmemload11:23:152
4005992213,7cyclictest13816-21cron12:40:012
400499223,5cyclictest0-21swapper/108:35:161
400499223,5cyclictest0-21swapper/108:35:151
4004992217,3cyclictest22-21ksoftirqd/107:20:001
400499221,4cyclictest22-21ksoftirqd/110:00:211
400499221,4cyclictest22-21ksoftirqd/110:00:201
400699211,7cyclictest7818-21cstates12:35:143
4006992112,7cyclictest11151-21users11:00:273
4006992112,6cyclictest8504-21tr10:35:133
400699211,11cyclictest13328-21cat09:25:273
400599214,3cyclictest23366-21diskmemload09:50:262
400599214,3cyclictest23366-21diskmemload09:22:452
400599213,3cyclictest23366-21diskmemload12:17:272
400599213,3cyclictest23366-21diskmemload12:13:522
400599213,3cyclictest23366-21diskmemload12:13:512
400599213,3cyclictest23366-21diskmemload11:14:262
400599213,3cyclictest23366-21diskmemload10:58:042
400599212,4cyclictest23366-21diskmemload10:11:162
4005992113,6cyclictest22292-21awk10:20:242
4005992110,8cyclictest560-21dbus-daemon11:05:232
400499212,5cyclictest22-21ksoftirqd/107:15:001
83402200,2sleep30-21swapper/310:59:233
400699209,9cyclictest5738-21cron09:45:003
400699205,6cyclictest34-21ksoftirqd/310:10:553
400699205,3cyclictest141rcu_preempt10:00:153
400699205,3cyclictest141rcu_preempt10:00:143
400699204,3cyclictest23366-21diskmemload12:12:553
400699204,3cyclictest23366-21diskmemload12:12:543
400699204,3cyclictest23366-21diskmemload10:44:203
400699202,3cyclictest141rcu_preempt11:50:133
400699201,6cyclictest30870-21apt-get11:40:133
400699201,6cyclictest28508-21apt-get10:50:153
4006992014,4cyclictest29685-21cpuspeed_turbos11:15:133
4006992013,4cyclictest23810-21sh09:10:133
4006992013,4cyclictest23366-21diskmemload11:31:363
4006992012,6cyclictest22581-21cat11:10:013
4006992012,6cyclictest20575-21cat09:55:203
4006992011,7cyclictest278-21systemd-journal12:30:013
400699201,15cyclictest52750irq/139-enp3s0-11:20:133
400699201,10cyclictest1298-21grep12:30:163
400699200,18cyclictest1887-21swap10:05:243
400599202,3cyclictest23366-21diskmemload11:25:112
400599202,3cyclictest23366-21diskmemload11:25:102
400599202,3cyclictest23366-21diskmemload09:30:532
400599201,5cyclictest0-21swapper/212:23:482
400699192,12cyclictest18454-21apt-get09:30:133
4006991914,3cyclictest25531-21switchtime11:35:233
4006991912,5cyclictest23807-21ntp_kernel_err11:10:213
4006991912,5cyclictest15444-21ls10:15:243
4006991911,6cyclictest14287-21ls09:50:253
4006991911,6cyclictest10632-21cat11:25:143
4006991911,6cyclictest10632-21cat11:25:133
400699191,15cyclictest7425-21mailstats09:45:223
4005991912,5cyclictest11477-21wc11:50:012
4006991812,4cyclictest636-21runrttasks09:17:493
4006991810,6cyclictest6254-21threads09:20:243
4006991810,5cyclictest19395-21missed_timers11:55:183
400599188,2cyclictest28-21ksoftirqd/207:14:572
400599182,3cyclictest292902ssh09:37:522
4006991712,3cyclictest7273-21sendmail_mailqu07:15:223
4006991710,5cyclictest4224-21cat07:10:133
400599179,5cyclictest8256-21cat07:20:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional