You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-02 - 19:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Tue Jun 02, 2026 12:45:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
43772163107,27sleep20-21swapper/207:07:472
42852163107,28sleep00-21swapper/007:06:370
45282162105,28sleep10-21swapper/107:09:431
44142162109,29sleep30-21swapper/307:08:163
281562970,5sleep012-21ksoftirqd/010:48:020
36152630,1sleep20-21swapper/210:05:272
218942540,1sleep1211rcuc/109:55:221
61512530,1sleep00-21swapper/012:07:290
317252480,2sleep10-21swapper/109:15:181
224042480,1sleep20-21swapper/211:07:322
114022450,1sleep20-21swapper/211:23:402
247612440,2sleep00-21swapper/009:57:430
305452430,2chrt0-21swapper/110:25:241
249232420,1sleep10-21swapper/112:21:061
117712390,1sleep10-21swapper/109:24:441
326952380,1sleep00-21swapper/011:39:310
482499362,24cyclictest14216-21latency_hist11:50:002
482499354,15cyclictest30786-21grep10:25:262
4825993327,3cyclictest294392sleep310:25:143
482499324,19cyclictest18447-21ssh11:28:592
482499323,15cyclictest25510-21ssh09:10:242
482499323,13cyclictest0-21swapper/210:13:392
482499323,10cyclictest14728-21sed08:45:242
482499322,9cyclictest0-21swapper/211:35:002
4823993217,12cyclictest8506-21grep08:30:251
482399321,23cyclictest26115-21cpuspeed_turbos09:35:131
482499315,12cyclictest16022-21tune2fs07:35:162
482499313,15cyclictest23974-21diskmemload11:04:242
482499313,15cyclictest0-21swapper/210:36:252
482499313,11cyclictest3508-21taskset10:30:122
482499312,16cyclictest26452-21tune2fs09:35:152
4823993127,3cyclictest52950irq/141-enp3s0-09:48:121
4823993127,3cyclictest52950irq/141-enp3s0-09:48:121
104382310,2sleep3331rcuc/309:23:263
482499305,9cyclictest8473-21sh12:33:322
482499305,8cyclictest27526-21rm11:10:532
482499304,9cyclictest23974-21diskmemload10:58:512
482499304,12cyclictest1983-21ls12:05:002
482499303,18cyclictest27613-21ssh12:23:512
482499303,11cyclictest23974-21diskmemload12:36:512
482499303,10cyclictest15973-21ssh11:50:232
482499302,16cyclictest0-21swapper/211:59:012
482499302,12cyclictest20118-21rm12:18:032
482499302,12cyclictest19143-21rm09:53:502
482499302,10cyclictest10620-21sed12:10:252
482399301,27cyclictest640-21snmpd09:41:261
155012300,1sleep30-21swapper/311:50:183
145762300,1sleep00-21swapper/010:14:360
482599291,26cyclictest34-21ksoftirqd/310:07:163
482499293,3cyclictest11047-21rm09:24:012
482499293,10cyclictest25566-21rm09:34:592
482499293,10cyclictest21878-21munin-run07:50:002
482499292,9cyclictest20265-21rm10:41:562
482499292,11cyclictest1091-21cat11:15:252
482499291,5cyclictest7768-21sed09:45:162
482499291,5cyclictest7768-21sed09:45:162
482399292,25cyclictest0-21swapper/110:05:441
482399290,26cyclictest16014-21ssh09:27:141
482299291,26cyclictest6005-21ssh10:07:490
482299291,25cyclictest0-21swapper/009:35:240
4825992822,3cyclictest21105-21python309:00:233
482599281,24cyclictest26377-21df09:35:133
482499284,10cyclictest15466-21rm09:26:412
482499283,9cyclictest0-21swapper/210:21:392
482499283,8cyclictest805-21rm10:51:102
482499283,16cyclictest5980-21ssh11:43:282
482499283,15cyclictest0-21swapper/212:26:222
482499283,10cyclictest30783-21date08:10:002
482499283,10cyclictest0-21swapper/209:00:212
482499282,10cyclictest0-21swapper/209:40:132
482399284,15cyclictest0-21swapper/109:30:431
4823992817,4cyclictest19400-21ssh11:29:561
482299281,3cyclictest0-21swapper/009:50:380
482299281,26cyclictest0-21swapper/009:49:520
482299281,26cyclictest0-21swapper/009:49:510
482299281,25cyclictest0-21swapper/008:55:300
482299280,26cyclictest52750irq/139-enp3s0-09:23:570
4825992723,2cyclictest7171-21apt-get09:45:113
4825992723,2cyclictest7171-21apt-get09:45:113
482599271,3cyclictest19168-21ssh09:53:523
482599271,24cyclictest6707-21ssh09:44:583
482499273,10cyclictest11244-21sed07:25:002
482499273,10cyclictest11244-21sed07:25:002
482499272,8cyclictest27426-21ssh10:47:192
482499272,8cyclictest19091-21ssh10:17:212
482499272,8cyclictest12297-21ntp_states08:40:212
482499272,8cyclictest0-21swapper/207:10:132
482499271,10cyclictest2335-21rm12:05:102
482399271,2cyclictest0-21swapper/108:40:181
482399271,24cyclictest0-21swapper/109:50:501
482399271,24cyclictest0-21swapper/109:14:201
482399271,19cyclictest0-21swapper/109:08:441
4822992724,2cyclictest0-21swapper/009:41:250
482299271,2cyclictest0-21swapper/009:34:390
482299271,24cyclictest0-21swapper/009:26:260
482299271,24cyclictest0-21swapper/008:29:450
482299270,25cyclictest0-21swapper/009:05:120
270652270,2chrt0-21swapper/010:23:180
4825992622,2cyclictest0-21swapper/309:27:373
4825992621,3cyclictest53050irq/142-enp3s0-09:32:353
482599261,23cyclictest0-21swapper/308:40:233
482499264,8cyclictest30722-21sh11:37:332
482499263,12cyclictest7081-21taskset08:30:122
482499262,9cyclictest23974-21diskmemload09:19:012
482499262,9cyclictest15535-21cat07:35:012
482499262,6cyclictest4683-21munin-run08:25:002
482499260,24cyclictest28-21ksoftirqd/209:08:262
482499260,10cyclictest23974-21diskmemload10:03:282
4823992623,2cyclictest0-21swapper/110:04:391
4823992623,2cyclictest0-21swapper/109:00:201
4823992617,3cyclictest882-21ssh10:51:141
482399261,1cyclictest0-21swapper/108:55:081
4822992624,1cyclictest0-21swapper/009:04:500
482299261,1cyclictest0-21swapper/010:04:500
122522260,2sleep30-21swapper/311:48:013
482599251,23cyclictest0-21swapper/309:05:013
482599251,1cyclictest0-21swapper/308:55:103
482499251,9cyclictest776-21cstates08:15:132
482499251,23cyclictest0-21swapper/209:55:162
482399251,1cyclictest0-21swapper/108:54:551
80072240,8sleep228-21ksoftirqd/207:15:202
4825992422,1cyclictest0-21swapper/309:55:253
4825992422,1cyclictest0-21swapper/308:53:013
4825992422,1cyclictest0-21swapper/308:30:243
482599241,1cyclictest0-21swapper/310:04:493
4825992411,9cyclictest34-21ksoftirqd/310:55:123
482499243,8cyclictest11768-21perl07:25:152
482499242,9cyclictest17765-21/usr/sbin/munin08:55:252
482499241,7cyclictest25570-21ls07:55:262
482499240,9cyclictest0-21swapper/208:50:222
4823992422,1cyclictest0-21swapper/108:29:441
4823992411,3cyclictest6355-21sh10:31:321
482299247,6cyclictest0-21swapper/011:28:340
4822992422,1cyclictest0-21swapper/008:40:200
265492240,2sleep10-21swapper/110:46:281
482399234,7cyclictest22-21ksoftirqd/112:30:251
4823992314,7cyclictest6191-21ntpq10:55:211
482299236,3cyclictest141rcu_preempt11:15:230
482299234,3cyclictest23974-21diskmemload11:07:330
4822992317,4cyclictest12-21ksoftirqd/012:00:010
4822992312,3cyclictest12-21ksoftirqd/010:30:240
4825992214,6cyclictest20733-21ls07:45:173
4825992210,9cyclictest34-21ksoftirqd/310:30:233
4825992210,9cyclictest34-21ksoftirqd/309:10:223
482499224,6cyclictest0-21swapper/208:25:252
482499222,6cyclictest18985-21tr07:40:242
482499221,9cyclictest640-21snmpd07:51:512
482399225,8cyclictest18773-21ssh10:17:021
482399225,5cyclictest22-21ksoftirqd/110:35:271
4823992214,6cyclictest18133-21pluginstate10:40:211
4823992212,7cyclictest7351-21cat07:15:141
482299224,10cyclictest12-21ksoftirqd/011:10:140
482299221,8cyclictest31540-21diskstats09:15:160
4822992211,3cyclictest12-21ksoftirqd/011:20:130
233662220,1sleep30-21swapper/311:55:453
482599219,2cyclictest34-21ksoftirqd/308:35:263
482599216,13cyclictest34-21ksoftirqd/312:20:223
482599213,10cyclictest141rcu_preempt10:40:193
482599211,9cyclictest239742diskmemload12:07:293
4825992113,6cyclictest5672-21cat08:25:203
4825992113,6cyclictest30982-21ls10:50:143
4825992112,7cyclictest14213-21ntpq11:25:203
4825992112,7cyclictest12215-21ntpq07:25:193
4825992112,6cyclictest31070-21cat08:10:113
482599211,14cyclictest25327-21ssh11:10:123
482399217,3cyclictest12615-21rm10:12:381
482399216,7cyclictest22-21ksoftirqd/111:40:061
482399216,7cyclictest0-21swapper/110:22:291
482399215,7cyclictest23974-21diskmemload11:14:501
482399213,2cyclictest141rcu_preempt11:20:001
4823992113,6cyclictest9512-21ls11:45:241
4823992113,6cyclictest1081-21perl08:15:151
4823992112,6cyclictest24792-21seq11:09:541
482399211,11cyclictest0-21swapper/112:37:231
482399210,19cyclictest15981-21ssh11:02:491
482299217,5cyclictest0-21swapper/012:36:320
482299215,4cyclictest66050lldpd08:52:390
482299214,4cyclictest23974-21diskmemload10:28:150
482299211,6cyclictest24646-21taskset12:20:500
482299211,5cyclictest0-21swapper/012:31:150
482299211,4cyclictest12599-21awk11:00:180
482299211,4cyclictest0-21swapper/012:27:430
265942210,3sleep3331rcuc/311:35:013
13152210,2sleep30-21swapper/312:28:143
482599208,9cyclictest34-21ksoftirqd/311:05:163
482599208,4cyclictest19401-1kworker/u9:011:40:133
482599204,8cyclictest141rcu_preempt08:05:143
482599202,12cyclictest21774-21apt-get10:20:033
482599201,6cyclictest32109-21ssh11:15:123
4825992012,6cyclictest24321-21fw_packets10:45:173
4825992012,6cyclictest22685-21smartctl07:50:173
4825992011,7cyclictest9311-21basename10:10:163
4825992011,6cyclictest24999-21cat07:55:193
482599200,5cyclictest0-21swapper/312:33:373
482499201,7cyclictest28-21ksoftirqd/208:02:262
482399208,3cyclictest141rcu_preempt12:05:181
482399205,7cyclictest10473-21ssh11:22:441
482399205,6cyclictest16043-21fschecks_time07:35:161
4823992015,4cyclictest22-21ksoftirqd/107:15:011
4823992013,5cyclictest636-21runrttasks12:15:321
4823992013,4cyclictest32550-21sh11:39:221
4823992013,4cyclictest31811-21ntp_offset08:10:181
482399201,2cyclictest141rcu_preempt07:25:241
4823992012,6cyclictest31671-21seq12:02:191
4823992012,6cyclictest10050-21ntp_kernel_err07:20:201
4823992012,6cyclictest10050-21ntp_kernel_err07:20:191
482399201,10cyclictest22-21ksoftirqd/108:45:161
4823992010,7cyclictest10368-21sensors_temp08:35:231
482299203,11cyclictest24264-21apt-get07:55:120
482299201,8cyclictest141rcu_preempt10:35:060
482299201,6cyclictest0-21swapper/010:56:100
4822992014,4cyclictest15362-21cron12:15:000
4822992013,4cyclictest23974-21diskmemload09:10:190
482299201,16cyclictest20953-21ntp_states11:30:200
4822992010,7cyclictest607-21apt-get08:15:110
482599197,8cyclictest13516-21apt-get08:45:133
482599191,5cyclictest532-21in:imuxsock07:25:003
482599191,5cyclictest532-21in:imuxsock07:25:003
4825991914,3cyclictest53050irq/142-enp3s0-11:00:133
4825991913,4cyclictest13639-21sh12:36:573
4825991912,5cyclictest17748-21sed07:40:013
4825991911,6cyclictest7199-21awk11:20:193
4825991911,6cyclictest20456-21expr11:30:163
4825991911,5cyclictest13930-21expr07:30:173
482599190,4cyclictest0-21swapper/307:17:533
482499191,6cyclictest28-21ksoftirqd/208:35:422
482499191,6cyclictest28-21ksoftirqd/208:10:142
482399194,7cyclictest19729-21ssh11:30:031
482399191,8cyclictest0-21swapper/112:10:131
482399191,2cyclictest141rcu_preempt08:10:001
482399191,2cyclictest141rcu_preempt07:55:231
4823991912,5cyclictest14934-21expr11:50:141
4823991911,6cyclictest30460-21which12:25:211
482399190,17cyclictest21615-21expr11:55:141
482299199,7cyclictest278-21systemd-journal11:50:000
482299199,6cyclictest52750irq/139-enp3s0-12:00:110
482299191,6cyclictest4380-21kworker/0:011:54:180
482299191,2cyclictest0-21swapper/010:40:200
4822991912,5cyclictest30013-21wc08:05:230
4822991912,5cyclictest16816-21wc10:15:240
4822991912,3cyclictest131rcuc/012:18:160
4822991911,6cyclictest27616-21grep08:00:220
4822991911,5cyclictest11848-21sed07:25:150
4822991910,7cyclictest30664-21ssh10:50:080
4822991910,6cyclictest1893-21ls11:40:170
482599186,9cyclictest34-21ksoftirqd/312:10:123
482599183,8cyclictest141rcu_preempt09:15:203
482599181,9cyclictest141rcu_preempt12:15:153
482599181,7cyclictest10773-21grep10:35:163
482599181,3cyclictest16757-21python310:15:243
4825991813,3cyclictest27929-21sh12:00:053
4825991811,5cyclictest19168-21expr07:40:273
4823991813,3cyclictest27390-21missed_timers08:00:191
4823991810,6cyclictest4324-21seq08:20:391
4823991810,6cyclictest22950-21needreboot07:50:191
482299182,11cyclictest30903-21apt-get08:10:090
4822991813,3cyclictest18324-21head07:40:170
4822991811,5cyclictest22060-21sed07:50:010
4822991811,5cyclictest14938-21awk07:30:260
482599176,7cyclictest4865-21apt-get07:10:123
4825991712,3cyclictest1869-21timerandwakeup08:15:233
482399179,6cyclictest15180-21seq07:33:121
4823991712,3cyclictest19959-21/usr/sbin/munin07:45:241
4823991710,5cyclictest19919-21ls07:45:011
482299179,6cyclictest9945-21sed07:20:190
482299179,6cyclictest9945-21sed07:20:180
482299179,6cyclictest7379-21expr08:30:130
4822991713,2cyclictest12125-21munin-node07:10:110
4822991711,4cyclictest23998-21cron07:54:590
4822991710,5cyclictest278-21systemd-journal07:20:000
4822991710,5cyclictest11105-21cat08:40:000
482599169,5cyclictest3300-21tail08:20:133
482599169,5cyclictest26809-21expr08:00:133
4822991611,4cyclictest12-21ksoftirqd/008:25:000
4822991611,3cyclictest636-21runrttasks08:45:010
482299157,6cyclictest15570-21cat07:35:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional