You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-16 - 21:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sat May 16, 2026 12:45:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6707221397,95sleep20-21swapper/207:05:062
84372196174,15sleep00-21swapper/007:07:450
84552142105,26sleep10-21swapper/107:07:591
85442139102,25sleep30-21swapper/307:09:093
87132740,3chrt0-21swapper/312:30:203
87132740,3chrt0-21swapper/312:30:193
214812710,2sleep122-21ksoftirqd/110:40:131
212932660,2sleep30-21swapper/312:15:223
9872640,3sleep2271rcuc/211:36:292
83592630,2sleep00-21swapper/011:42:050
227682570,2sleep20-21swapper/211:29:192
138862550,2sleep20-21swapper/210:58:242
326492520,2sleep20-21swapper/209:12:382
222902520,2sleep10-21swapper/111:28:481
136022510,2sleep10-21swapper/110:34:271
127292510,3sleep0131rcuc/010:57:140
315902500,1sleep00-21swapper/008:00:210
15682490,1sleep00-21swapper/011:37:030
32632470,2sleep30-21swapper/308:10:193
117312460,1sleep211730-21ssh12:08:522
51202450,2sleep228-21ksoftirqd/212:04:092
322752440,3sleep00-21swapper/009:12:160
163452430,1sleep10-21swapper/110:35:281
45802420,2chrt0-21swapper/210:50:502
137352370,2chrt0-21swapper/209:45:362
228082350,3sleep0131rcuc/010:40:240
242612300,2sleep30-21swapper/309:54:243
270082270,2chrt0-21swapper/209:05:192
264592270,8sleep012-21ksoftirqd/011:07:280
166472270,2sleep30-21swapper/311:24:533
888999261,24cyclictest0-21swapper/207:12:412
138082260,2sleep10-21swapper/111:21:591
889099255,17cyclictest23823-21head11:05:223
8890992511,12cyclictest22765-21cat08:55:183
8889992516,6cyclictest19984-21ssh12:15:112
8888992523,1cyclictest0-21swapper/107:12:151
888799257,8cyclictest278-21systemd-journal10:50:000
8887992513,6cyclictest7841-21apt-get12:30:130
8887992513,6cyclictest7841-21apt-get12:30:130
889099241,20cyclictest10056-21/usr/sbin/munin09:20:173
888999247,5cyclictest0-21swapper/209:43:122
8889992413,8cyclictest14140-21cron12:35:002
8889992413,8cyclictest14140-21cron12:34:592
8888992414,7cyclictest12200-21df09:45:151
8888992410,9cyclictest6963-21df12:05:141
235822240,1sleep20-21swapper/210:17:192
228402240,2sleep20-21swapper/207:40:162
179582240,2sleep00-21swapper/012:13:230
889099237,4cyclictest141rcu_preempt12:00:163
889099231,6cyclictest21716-21df_inode10:40:143
8890992314,6cyclictest52750irq/139-enp3s0-10:55:143
8890992313,8cyclictest669-21basename10:25:013
888999239,4cyclictest28-21ksoftirqd/210:30:002
8889992316,3cyclictest28-21ksoftirqd/209:20:262
888999231,5cyclictest0-21swapper/209:16:472
888999230,9cyclictest15024-21ssh12:10:252
8888992312,9cyclictest7609-21gpgconf12:30:011
8888992312,9cyclictest7609-21gpgconf12:30:001
888799232,15cyclictest18671-21ssh11:50:120
8887992310,9cyclictest12-21ksoftirqd/011:30:110
75952230,2sleep00-21swapper/010:30:000
308402230,1sleep00-21swapper/010:22:470
289982230,2sleep20-21swapper/207:55:182
889099229,4cyclictest141rcu_preempt07:55:213
889099228,5cyclictest34-21ksoftirqd/309:25:133
889099228,5cyclictest34-21ksoftirqd/309:25:123
889099227,6cyclictest34-21ksoftirqd/309:16:483
889099224,16cyclictest26362-21ntpq11:55:203
889099224,16cyclictest10193-21latency_hist11:20:013
889099221,6cyclictest16298-21users12:35:253
8890992216,3cyclictest14180-21dump-pmu-power10:35:003
8890992211,7cyclictest11839-21apt-get09:45:143
8890992211,4cyclictest141rcu_preempt10:04:423
8889992213,7cyclictest20563-21sed07:35:202
8889992213,5cyclictest28252-21diskmemload11:05:132
8888992216,4cyclictest22-21ksoftirqd/109:40:011
8888992213,7cyclictest29944-21seq09:58:181
8888992213,7cyclictest21798-21cron08:55:011
888799223,16cyclictest12-21ksoftirqd/012:05:130
888799221,7cyclictest27670-21taskset12:20:210
8887992214,6cyclictest560-21dbus-daemon11:20:010
8887992212,7cyclictest560-21dbus-daemon08:40:000
888799221,11cyclictest10910-21diskstats11:20:150
201312220,1sleep00-21swapper/009:26:570
201312220,1sleep00-21swapper/009:26:560
889099219,3cyclictest34-21ksoftirqd/307:35:243
889099213,2cyclictest141rcu_preempt11:30:253
889099211,8cyclictest117302ssh12:08:513
889099211,7cyclictest25351-21users09:30:263
889099211,6cyclictest0-21swapper/309:59:113
8890992113,6cyclictest7581-21pluginstate10:05:233
8890992113,6cyclictest278-21systemd-journal10:30:013
889099211,2cyclictest141rcu_preempt08:45:003
8890992112,7cyclictest14095-21ls12:10:183
8890992112,6cyclictest17762-21cat08:45:103
8890992112,6cyclictest1412-21awk08:05:223
889099210,8cyclictest3318-21idleruntime-cro11:15:003
889099210,7cyclictest28149-21ls12:20:263
888999218,9cyclictest948-21ssh12:25:132
8889992116,3cyclictest28-21ksoftirqd/211:20:002
8889992113,6cyclictest618-21ssh10:25:002
8889992112,6cyclictest560-21dbus-daemon12:25:002
8889992111,7cyclictest1-21systemd11:50:072
888899219,10cyclictest30310-21ssh11:10:161
8888992113,6cyclictest7182-21df_inode08:20:131
8888992112,7cyclictest32415-21seq08:03:061
8888992112,7cyclictest27080-21tune2fs12:20:151
8888992112,7cyclictest23647-21seq07:40:341
8888992111,7cyclictest21098-21threads07:35:251
8888992111,7cyclictest17305-21cut09:25:151
8888992111,7cyclictest17305-21cut09:25:151
888799219,9cyclictest12-21ksoftirqd/012:35:110
888799219,9cyclictest12-21ksoftirqd/009:45:140
888799219,9cyclictest12-21ksoftirqd/009:35:130
888799214,14cyclictest17269-21mailstats11:00:230
888799214,14cyclictest17269-21mailstats11:00:220
888799213,13cyclictest3310-21ssh09:15:120
888799211,6cyclictest624-21gdbus11:10:200
8887992115,3cyclictest28252-21diskmemload12:28:540
888799211,4cyclictest0-21swapper/010:09:100
8887992114,4cyclictest25535-21cstates11:55:130
8890992014,4cyclictest16823-21sh10:12:193
889099201,3cyclictest141rcu_preempt08:35:123
8890992012,6cyclictest8932-21cat10:55:003
8890992012,6cyclictest24610-21forks07:45:173
8890992012,6cyclictest24610-21forks07:45:163
8890992012,6cyclictest20457-21tune2fs10:15:143
8890992012,6cyclictest15098-21tune2fs10:35:173
8890992012,6cyclictest11830-21expr08:30:153
8890992012,5cyclictest29106-21grep09:10:153
889099200,5cyclictest0-21swapper/310:46:253
888999209,4cyclictest141rcu_preempt09:35:172
888999203,10cyclictest28130-21ssh10:45:122
888999202,12cyclictest6479-21apt-get10:05:142
888999202,12cyclictest16847-21apt-get09:25:102
888999202,12cyclictest16847-21apt-get09:25:102
888999201,17cyclictest22607-21mailstats10:40:232
8889992011,6cyclictest7412-21smartctl08:20:162
8889992011,2cyclictest141rcu_preempt10:34:062
8889992010,7cyclictest31163-21grep08:00:162
888899209,7cyclictest52850irq/140-enp3s0-09:15:131
888899208,8cyclictest13235-21apt-get07:20:121
888899202,12cyclictest636-21runrttasks08:30:121
8888992014,4cyclictest13901-21sh10:58:241
8888992012,6cyclictest4441-21basename08:15:001
8888992012,6cyclictest26368-21ntpq11:55:191
8888992012,6cyclictest21053-21pluginstate10:15:211
8888992012,6cyclictest13825-21diskstats08:35:151
8888992011,6cyclictest18400-21cut07:30:201
8888992011,6cyclictest1817-21grep08:05:261
8888992011,6cyclictest12291-21cat07:15:241
888899201,15cyclictest52850irq/140-enp3s0-10:20:111
888799203,12cyclictest13213-21ssh10:10:130
888799202,6cyclictest12-21ksoftirqd/009:50:070
888799202,3cyclictest141rcu_preempt09:55:300
888799202,13cyclictest11348-21apt-get08:30:110
888799201,6cyclictest16054-21uniq08:40:140
8887992015,3cyclictest28252-21diskmemload12:04:480
8887992012,6cyclictest6588-21sed09:40:240
8887992012,6cyclictest15916-21cat10:35:230
8887992011,6cyclictest8238-21grep08:20:240
8887992011,6cyclictest2832-21expr08:10:150
889099197,4cyclictest52750irq/139-enp3s0-11:38:053
889099192,3cyclictest28252-21diskmemload11:25:163
889099191,5cyclictest28252-21diskmemload11:45:023
889099191,5cyclictest28252-21diskmemload11:45:013
889099191,5cyclictest0-21swapper/309:35:173
889099191,4cyclictest0-21swapper/309:40:383
8890991914,3cyclictest1812-21missed_timers12:25:203
8890991913,4cyclictest7836-21head08:20:203
8890991913,4cyclictest26688-21sh09:05:163
8890991913,4cyclictest20947-21sh11:50:593
8890991912,5cyclictest6221-21rt-features11:40:233
8890991911,6cyclictest5293-21cat08:15:183
8890991911,6cyclictest22427-21cat07:40:133
8890991911,6cyclictest16393-21expr11:00:153
8890991911,6cyclictest16393-21expr11:00:153
8890991911,6cyclictest15867-21cat07:25:143
8890991911,6cyclictest14525-21cat07:20:243
8890991911,5cyclictest20825-21ntpq08:50:203
888999199,7cyclictest660-21lldpd08:25:252
888999198,2cyclictest28-21ksoftirqd/209:30:122
888999192,4cyclictest11304-21apt-get11:45:022
888999192,4cyclictest11304-21apt-get11:45:012
8889991913,4cyclictest8744-21kworker/u8:110:10:222
8889991912,5cyclictest31008-21grep11:10:232
8889991912,5cyclictest26784-21cut07:50:152
8889991912,5cyclictest24230-21expr11:30:132
8889991911,6cyclictest24889-21cut09:00:182
8889991911,6cyclictest16393-21sed07:25:192
8889991910,5cyclictest52950irq/141-enp3s0-10:35:122
888999190,12cyclictest141rcu_preempt09:57:432
8888991914,3cyclictest15526-21sh07:25:011
8888991913,4cyclictest6636-21sh12:29:281
8888991913,4cyclictest3801-21sh11:39:171
8888991913,4cyclictest25878-21sh09:30:551
8888991912,5cyclictest30027-21smartctl09:10:231
8888991912,5cyclictest27466-21rt-features07:50:231
8888991912,4cyclictest28252-21diskmemload12:14:351
8888991912,4cyclictest28252-21diskmemload10:51:471
8888991911,6cyclictest8704-21seq09:42:211
8888991911,6cyclictest29077-21uname10:45:211
8888991911,6cyclictest24283-21cat09:00:131
8888991911,6cyclictest2111-21ntpq10:25:201
8888991911,6cyclictest14053-21cut10:10:181
8888991911,5cyclictest28252-21diskmemload12:19:361
8888991910,6cyclictest25348-21timerandwakeup11:30:241
888799196,2cyclictest12-21ksoftirqd/010:19:020
888799192,4cyclictest12-21ksoftirqd/011:29:330
888799191,5cyclictest0-21swapper/012:16:130
888799191,4cyclictest28252-21diskmemload09:22:580
8887991913,4cyclictest9604-21vmstat10:30:280
8887991911,5cyclictest19205-21grep08:45:250
888799191,15cyclictest1291-21ssh10:00:260
8887991910,6cyclictest560-21dbus-daemon07:25:020
161722190,5sleep2271rcuc/208:40:152
8890991811,5cyclictest9164-21expr07:10:133
8890991811,5cyclictest26676-21expr07:50:143
8890991811,5cyclictest11293-21expr07:15:143
8890991810,6cyclictest31025-21sh08:00:153
888999188,7cyclictest52950irq/141-enp3s0-11:00:132
888999188,7cyclictest52950irq/141-enp3s0-11:00:122
888999187,3cyclictest141rcu_preempt08:10:232
888999182,11cyclictest5139-21cpuspeed_turbos11:40:132
888999181,4cyclictest0-21swapper/208:30:232
888999181,3cyclictest141rcu_preempt08:35:262
8889991813,3cyclictest15391-21latency12:35:172
8889991813,3cyclictest11158-21sh11:20:172
8889991812,4cyclictest28252-21diskmemload10:00:402
8889991810,6cyclictest560-21dbus-daemon07:50:002
8889991810,6cyclictest560-21dbus-daemon07:50:002
888899189,6cyclictest52850irq/140-enp3s0-12:00:111
888899189,6cyclictest20122-21cat09:50:251
888899189,5cyclictest52850irq/140-enp3s0-11:50:131
888899188,7cyclictest52850irq/140-enp3s0-12:35:131
8888991813,3cyclictest4490-21kernelversion11:15:171
8888991813,3cyclictest27442-21sh11:08:281
8888991813,3cyclictest24766-21gltestperf07:45:181
8888991813,3cyclictest24766-21gltestperf07:45:171
8888991813,3cyclictest10201-21cut09:20:131
8888991811,5cyclictest22-21ksoftirqd/111:40:231
8888991811,5cyclictest17438-21seq08:43:591
8888991811,4cyclictest28252-21diskmemload10:05:201
888899181,14cyclictest12456-21ssh11:45:161
888899181,14cyclictest12456-21ssh11:45:151
8888991810,6cyclictest28598-21cat07:55:141
8888991810,6cyclictest26641-21expr09:05:161
8888991810,6cyclictest2240-21seq10:01:161
8888991810,6cyclictest18677-21ntpq08:45:191
8888991810,6cyclictest15990-21cpu11:00:131
8888991810,6cyclictest15990-21cpu11:00:121
8888991810,6cyclictest10131-21run-parts08:25:241
888799186,7cyclictest12-21ksoftirqd/009:00:120
888799185,11cyclictest2708-21kworker/0:111:45:170
888799185,11cyclictest2708-21kworker/0:111:45:160
8887991811,5cyclictest12-21ksoftirqd/007:40:100
8887991811,4cyclictest23603-21sh09:30:120
888799181,14cyclictest18264-21gltestperf07:30:170
8887991810,6cyclictest24143-21wc09:00:000
889099179,6cyclictest9110-21expr08:25:143
889099179,6cyclictest24480-21cat09:00:143
889099172,10cyclictest17722-21apt-get07:30:143
8889991712,3cyclictest25302-21/usr/sbin/munin11:55:142
8889991711,4cyclictest15127-21dump-pmu-power07:25:012
888999171,14cyclictest0-21swapper/208:45:152
8889991710,5cyclictest346-21sort08:05:132
8889991710,5cyclictest22586-21gltestperf08:55:172
8889991710,5cyclictest20906-21awk08:50:212
8889991710,5cyclictest19629-21uname09:50:212
8888991712,3cyclictest23947-21sendmail09:00:001
888799179,6cyclictest20289-21tune2fs07:35:170
888799179,5cyclictest1074-21kernelversion08:05:190
888799178,6cyclictest4694-21apt-get08:15:130
888799178,6cyclictest278-21systemd-journal08:29:590
888799177,7cyclictest560-21dbus-daemon07:25:000
888799176,9cyclictest53050irq/142-enp3s0-08:50:140
888799174,4cyclictest141rcu_preempt09:05:130
8887991711,4cyclictest3247-21sed10:50:170
8887991710,5cyclictest25881-21seq07:46:490
8887991710,5cyclictest25881-21seq07:46:480
888999168,6cyclictest17697-21/usr/sbin/munin07:30:262
888799168,6cyclictest30436-21basename08:00:010
8887991611,3cyclictest12168-21mailstats07:15:240
888999152,10cyclictest12970-21munin-run07:20:002
8889991510,3cyclictest6005-21timerwakeupswit08:15:252
888899157,6cyclictest5290-21cat08:15:191
888799159,4cyclictest10137-21sort07:10:230
888799158,5cyclictest26385-21basename07:50:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional