You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-24 - 21:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Mon Aug 24, 2026 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
199402203110,28sleep20-21swapper/207:09:412
199312187121,33sleep00-21swapper/007:09:350
196242173101,29sleep10-21swapper/107:05:331
198202171101,27sleep30-21swapper/307:08:093
252412550,2sleep10-21swapper/110:34:191
292352540,3sleep1211rcuc/111:00:251
70562530,1sleep00-21swapper/011:08:380
2023899532,31cyclictest0-21swapper/007:30:100
89912520,2sleep3331rcuc/312:21:333
48632520,1sleep10-21swapper/111:06:271
301132520,1sleep00-21swapper/011:01:000
114012520,2sleep228-21ksoftirqd/208:00:242
79552510,2sleep1211rcuc/112:20:311
250612500,2sleep00-21swapper/010:57:420
145232500,1sleep20-21swapper/212:01:382
190282480,2sleep30-21swapper/312:29:513
175222480,2sleep10-21swapper/110:28:221
38132470,5chrt0-21swapper/011:30:130
2023899473,22cyclictest0-21swapper/011:45:110
20238994713,22cyclictest18842-21sh11:16:580
2023899470,34cyclictest14733-21ssh11:14:340
2023899457,22cyclictest16228-21seq08:12:460
2023899451,27cyclictest11227-21/usr/sbin/munin10:00:130
2023899451,26cyclictest9219-21ssh11:58:040
2023899451,24cyclictest31232-21taskset08:47:290
2023899450,2cyclictest7597-21diskmemload10:50:220
89772440,1sleep20-21swapper/209:58:162
285542440,1sleep00-21swapper/008:40:220
2023899441,24cyclictest31073-21ssh12:14:440
2023899440,31cyclictest24296-21ssh10:33:230
2023899431,29cyclictest29455-21ssh09:49:500
2023899431,23cyclictest10432-21ssh12:22:590
2023899431,1cyclictest0-21swapper/008:32:160
20238994310,15cyclictest28972-21sh12:36:170
2023899421,25cyclictest7597-21diskmemload10:10:110
2023899421,23cyclictest24057-21ssh12:33:080
2023899420,30cyclictest9271-21ssh10:45:320
2023899420,28cyclictest992-21ssh11:51:340
2023899420,27cyclictest17865-21sh10:28:420
2023899420,26cyclictest10579-21sh09:11:130
2023899420,25cyclictest29552-21rm10:36:520
2023899420,22cyclictest31263-21grep07:35:140
124112420,11sleep228-21ksoftirqd/211:35:492
2023899412,20cyclictest0-21swapper/010:05:130
2023899411,29cyclictest7597-21diskmemload09:41:360
2023899411,26cyclictest12448-21sh09:36:280
2023899411,25cyclictest3542-21seq09:00:090
2023899411,25cyclictest1120-21ssh09:28:510
2023899411,24cyclictest6246-21rm09:55:340
2023899411,24cyclictest25110-21ssh09:22:090
2023899411,23cyclictest3698-21ssh10:18:170
2023899410,24cyclictest6695-21rm10:44:430
2023899410,24cyclictest6695-21rm10:44:430
2023899410,24cyclictest17405-21ssh09:16:170
2023899410,23cyclictest6883-21ssh09:32:390
2023899410,23cyclictest32391-21ssh11:26:500
185312410,1sleep10-21swapper/111:40:251
210372400,2sleep30-21swapper/312:06:233
2023899401,26cyclictest13768-21sh12:00:510
2023899401,22cyclictest8901-21sh10:21:430
2023899401,22cyclictest1744-21sh09:52:500
296812390,1sleep30-21swapper/310:13:393
286922390,9sleep334-21ksoftirqd/310:36:033
2023899391,27cyclictest21525-21sh11:43:100
2023899391,22cyclictest19228-21fschecks_count08:20:140
2023899391,21cyclictest7597-21diskmemload12:15:500
2023899391,21cyclictest7597-21diskmemload12:15:490
2023899391,21cyclictest18684-21ssh12:29:310
2023899391,19cyclictest0-21swapper/012:05:120
2023899390,28cyclictest25155-21ssh11:21:320
2023899390,20cyclictest7640-21cat07:55:000
2023899381,22cyclictest7597-21diskmemload11:38:010
2023899381,22cyclictest24702-21cat07:20:000
2023899370,20cyclictest7650-21gpgv07:55:000
2023899362,17cyclictest1049-21apt-get07:40:090
2023899361,21cyclictest636-21runrttasks08:15:550
2023899361,20cyclictest0-21swapper/008:50:140
2023899360,21cyclictest26411-21mailstats08:35:230
2023899351,18cyclictest21417-21perl08:25:170
2023899350,20cyclictest5686-21/usr/sbin/munin09:05:190
2023899341,20cyclictest12134-21latency_hist08:05:000
2023899341,17cyclictest0-21swapper/007:45:130
2023899340,21cyclictest3074-21taskset08:57:160
2023999331,29cyclictest5686-21/usr/sbin/munin09:05:211
2023899321,29cyclictest0-21swapper/008:07:410
20239993123,6cyclictest1826-21missed_timers07:40:171
2023899311,27cyclictest26192-21unixbench-2d07:20:260
2023899311,1cyclictest0-21swapper/007:25:590
2023899310,19cyclictest20002-21perf07:14:550
25552290,2sleep10-21swapper/112:16:541
25552290,2sleep10-21swapper/112:16:531
2024199298,8cyclictest34-21ksoftirqd/312:19:373
2024199298,8cyclictest34-21ksoftirqd/312:19:373
20241992913,9cyclictest34-21ksoftirqd/309:54:463
188442290,9sleep122-21ksoftirqd/110:53:131
2024199286,11cyclictest34-21ksoftirqd/311:50:063
2024199286,10cyclictest331rcuc/309:28:403
2024199283,8cyclictest34-21ksoftirqd/309:20:123
2024199282,3cyclictest34-21ksoftirqd/307:20:213
2024199281,25cyclictest0-21swapper/308:00:123
20241992810,4cyclictest34-21ksoftirqd/312:39:173
2024099281,25cyclictest0-21swapper/208:19:372
2023999281,25cyclictest0-21swapper/109:00:101
2023999281,25cyclictest0-21swapper/108:36:561
2023999281,25cyclictest0-21swapper/108:25:091
2023999281,25cyclictest0-21swapper/108:00:591
2023999281,25cyclictest0-21swapper/107:51:541
2023999281,19cyclictest0-21swapper/107:35:151
12822280,2sleep30-21swapper/311:04:353
44652270,4sleep34463-21ssh09:30:253
2024199277,9cyclictest34-21ksoftirqd/311:36:193
2024199276,9cyclictest34-21ksoftirqd/309:59:203
20241992722,3cyclictest7438-21apt-get07:55:033
20241992721,4cyclictest331rcuc/310:49:593
20241992718,7cyclictest26531-21ssh10:59:113
20241992714,4cyclictest34-21ksoftirqd/309:16:273
20241992713,4cyclictest34-21ksoftirqd/311:21:523
2024199271,24cyclictest0-21swapper/309:05:173
2024199271,24cyclictest0-21swapper/308:41:253
2024199271,24cyclictest0-21swapper/307:54:533
2024199271,10cyclictest34-21ksoftirqd/311:07:413
20240992722,3cyclictest29880-21df08:45:152
2024099271,3cyclictest640-21snmpd07:32:362
2024099271,3cyclictest636-21runrttasks08:11:172
2024099271,24cyclictest0-21swapper/208:23:242
2024099271,24cyclictest0-21swapper/207:49:072
2024099271,24cyclictest0-21swapper/207:42:502
2024099270,25cyclictest0-21swapper/209:07:402
2023999271,2cyclictest0-21swapper/107:45:591
2023999271,24cyclictest0-21swapper/108:45:231
2023999271,24cyclictest0-21swapper/108:42:211
2023999271,24cyclictest0-21swapper/108:30:051
2023999271,24cyclictest0-21swapper/108:16:291
2023999271,24cyclictest0-21swapper/108:12:421
2023999271,24cyclictest0-21swapper/108:05:551
2023999271,24cyclictest0-21swapper/107:55:281
2023999271,24cyclictest0-21swapper/107:31:241
2024199265,8cyclictest34-21ksoftirqd/311:15:123
2024199265,10cyclictest34-21ksoftirqd/312:13:463
2024199264,9cyclictest34-21ksoftirqd/311:57:103
2024199263,13cyclictest34-21ksoftirqd/309:12:573
2024199262,6cyclictest5059-21ls11:30:253
20241992622,2cyclictest0-21swapper/308:23:453
2024199262,16cyclictest16421-21ssh09:40:133
2024199261,23cyclictest0-21swapper/309:00:353
2024199261,23cyclictest0-21swapper/308:56:013
2024199261,23cyclictest0-21swapper/308:36:383
2024199261,23cyclictest0-21swapper/308:15:283
2024199261,23cyclictest0-21swapper/308:10:003
2024199261,23cyclictest0-21swapper/308:07:023
2024199261,23cyclictest0-21swapper/307:45:543
2024199261,23cyclictest0-21swapper/307:40:053
2024199261,23cyclictest0-21swapper/307:36:373
2024199261,23cyclictest0-21swapper/307:25:493
20241992611,4cyclictest34-21ksoftirqd/312:03:293
2024099261,23cyclictest0-21swapper/209:10:072
2024099261,23cyclictest0-21swapper/209:00:032
2024099261,23cyclictest0-21swapper/208:57:022
2024099261,23cyclictest0-21swapper/208:40:452
2024099261,23cyclictest0-21swapper/208:35:282
2024099261,23cyclictest0-21swapper/208:31:082
2024099261,23cyclictest0-21swapper/208:25:372
2023999261,24cyclictest0-21swapper/108:50:241
2023999261,24cyclictest0-21swapper/108:21:271
2023999261,24cyclictest0-21swapper/107:25:331
2023999260,24cyclictest0-21swapper/108:55:071
2023999260,24cyclictest0-21swapper/107:24:031
150642260,11sleep228-21ksoftirqd/210:25:542
2024199258,8cyclictest141rcu_preempt10:52:443
2024199254,12cyclictest34-21ksoftirqd/309:47:183
2024199254,11cyclictest34-21ksoftirqd/312:32:303
20241992523,1cyclictest0-21swapper/308:50:223
20241992523,1cyclictest0-21swapper/308:46:163
20241992520,4cyclictest331rcuc/311:10:273
20241992512,4cyclictest34-21ksoftirqd/310:44:043
20241992512,4cyclictest34-21ksoftirqd/310:44:033
2024199251,23cyclictest0-21swapper/307:30:563
2024199251,11cyclictest34-21ksoftirqd/311:26:113
2024199250,23cyclictest0-21swapper/308:30:383
20240992523,1cyclictest0-21swapper/207:25:032
20240992522,2cyclictest0-21swapper/207:56:412
20240992517,6cyclictest4440-21ls11:55:012
2024099251,23cyclictest0-21swapper/208:51:392
2024099251,23cyclictest0-21swapper/207:35:252
2024099250,23cyclictest0-21swapper/208:05:222
2024099250,23cyclictest0-21swapper/207:50:342
20239992523,1cyclictest0-21swapper/109:13:481
147912250,1sleep20-21swapper/209:15:112
2024199249,8cyclictest34-21ksoftirqd/310:19:173
2024199248,9cyclictest34-21ksoftirqd/311:41:013
2024199248,9cyclictest34-21ksoftirqd/308:28:253
2024199247,4cyclictest34-21ksoftirqd/311:47:543
2024199245,9cyclictest34-21ksoftirqd/310:26:023
20241992419,4cyclictest34-21ksoftirqd/310:10:013
20240992422,1cyclictest0-21swapper/207:23:062
18962241,20sleep11912-21sh09:53:001
2024199234,9cyclictest34-21ksoftirqd/310:23:203
2024199234,11cyclictest34-21ksoftirqd/307:12:253
20241992321,1cyclictest0-21swapper/307:19:593
20241992318,3cyclictest331rcuc/310:34:013
20241992311,4cyclictest34-21ksoftirqd/310:02:383
20241992311,4cyclictest34-21ksoftirqd/309:39:423
20240992311,10cyclictest718-21head10:15:242
20239992315,6cyclictest22-21ksoftirqd/109:40:001
20239992314,7cyclictest25201-21seq09:22:161
20239992314,6cyclictest19727-21sed12:05:241
78782220,1sleep10-21swapper/109:57:121
222172220,2chrt0-21swapper/211:43:522
2024099222,7cyclictest10417-21runrttasks12:22:582
20240992216,4cyclictest30142-21ssh12:13:492
2023999226,8cyclictest20108-21ssh10:05:521
20239992213,7cyclictest4599-21ntpq11:30:201
20239992213,7cyclictest22405-21smartctl10:55:241
20239992212,8cyclictest640-21snmpd09:17:551
20239992212,6cyclictest53050irq/142-enp3s0-10:10:121
20240992116,4cyclictest28-21ksoftirqd/211:24:592
20240992115,4cyclictest28-21ksoftirqd/211:15:002
20240992114,4cyclictest5350-21grep11:55:162
20239992113,6cyclictest16446-21memory11:15:181
20239992110,3cyclictest22-21ksoftirqd/112:15:011
20239992110,3cyclictest22-21ksoftirqd/111:25:151
70552200,2sleep20-21swapper/211:32:122
2024099206,5cyclictest7248-21sort10:20:232
20240992015,3cyclictest26885-21gltestperf10:35:152
20240992014,4cyclictest21217-21ls12:30:262
20240992012,6cyclictest32227-21seq09:50:532
20240992012,6cyclictest23724-21df09:45:142
20240992012,6cyclictest20886-21if_enp3s007:10:172
20240992011,7cyclictest560-21dbus-daemon12:30:002
20240992011,7cyclictest32539-21uname12:15:192
20240992011,7cyclictest32539-21uname12:15:192
2023999209,7cyclictest53050irq/142-enp3s0-11:45:111
20239992014,4cyclictest5841-21acpi10:20:111
20239992014,4cyclictest25110-21sh11:21:301
20239992014,4cyclictest11431-21sh11:11:171
20239992013,5cyclictest26531-21cpuspeed_turbos12:35:141
20239992011,7cyclictest682-21find10:40:121
20239992011,7cyclictest682-21find10:40:111
2024099199,6cyclictest52850irq/140-enp3s0-10:40:252
2024099199,6cyclictest52850irq/140-enp3s0-10:40:242
2024099199,3cyclictest141rcu_preempt10:46:022
2024099194,12cyclictest29254-21ssh11:25:122
2024099191,7cyclictest14168-21ssh10:50:042
2024099191,5cyclictest0-21swapper/211:15:152
2024099191,5cyclictest0-21swapper/209:31:352
20240991913,4cyclictest29442-21sh11:49:202
20240991913,4cyclictest23279-21interrupts07:15:162
20240991912,5cyclictest30838-21seq09:26:092
20240991911,6cyclictest2425-21cat11:05:152
20240991911,6cyclictest18730-21sed10:05:192
20240991911,6cyclictest12846-21head10:00:262
20240991911,5cyclictest278-21systemd-journal09:45:002
2024099190,4cyclictest0-21swapper/210:30:172
2023999192,1cyclictest141rcu_preempt10:00:191
2023999192,12cyclictest22632-21apt-get07:15:101
20239991914,3cyclictest32289-21vmstat11:50:271
20239991913,4cyclictest17708-21sh12:04:491
20239991913,4cyclictest10111-21sh10:46:241
20239991912,5cyclictest22-21ksoftirqd/111:40:011
20239991911,6cyclictest24859-21expr09:45:261
20239991911,6cyclictest18928-21seq09:41:131
20239991911,5cyclictest53050irq/142-enp3s0-10:15:081
20239991910,7cyclictest27254-21cat10:35:191
20239991910,7cyclictest16855-21rm12:27:411
2024099187,9cyclictest281ksoftirqd/212:40:012
2024099181,4cyclictest0-21swapper/212:06:132
20240991813,3cyclictest387-21sh11:03:452
20240991813,3cyclictest26450-21sh10:10:262
20240991813,3cyclictest24995-21sh10:57:372
20240991811,5cyclictest23105-21sed09:20:232
20240991810,6cyclictest10313-21cat09:35:152
2023999183,4cyclictest22-21ksoftirqd/109:32:041
20239991813,3cyclictest4838-21acpi11:55:121
20239991812,4cyclictest30829-21sh09:26:071
20239991812,4cyclictest22258-21sh12:31:201
2023999179,6cyclictest20446-21cat07:10:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional