You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-12 - 16:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sun Jul 12, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3909210188,9sleep20-21swapper/207:06:062
413729272,15sleep00-21swapper/007:09:010
419028661,9sleep30-21swapper/307:09:443
420028059,6sleep10-21swapper/107:09:501
176162550,0sleep10-21swapper/111:37:031
430899290,3cyclictest29473-21diskmemload11:45:233
430599280,28cyclictest0-21swapper/212:01:422
430899274,22cyclictest29473-21diskmemload10:50:123
430899270,26cyclictest0-21swapper/312:20:143
430899270,26cyclictest0-21swapper/312:13:523
4303992712,6cyclictest25-21ksoftirqd/110:28:211
430899262,21cyclictest0-21swapper/310:46:363
430899262,21cyclictest0-21swapper/310:46:353
430399261,24cyclictest0-21swapper/110:18:321
430899251,22cyclictest29473-21diskmemload12:02:323
430399252,22cyclictest29473-21diskmemload10:08:051
430899248,14cyclictest3833-21apache211:22:403
430899243,20cyclictest29473-21diskmemload10:08:413
430899243,20cyclictest19442-21lockfile-create11:00:013
430899240,17cyclictest0-21swapper/312:16:073
430599240,24cyclictest0-21swapper/209:18:002
4303992412,4cyclictest38150irq/131-snd_hda11:16:481
430899231,20cyclictest29473-21diskmemload11:06:553
430899230,3cyclictest0-21swapper/311:00:133
430899230,22cyclictest0-21swapper/311:26:193
430899230,1cyclictest0-21swapper/312:35:193
430899230,1cyclictest0-21swapper/310:39:153
430899230,1cyclictest0-21swapper/310:22:043
430899230,1cyclictest0-21swapper/309:42:043
430899230,19cyclictest29473-21diskmemload12:32:363
430899230,16cyclictest0-21swapper/311:40:033
430599230,0cyclictest0-21swapper/211:34:222
430899223,16cyclictest29473-21diskmemload11:35:263
430899221,20cyclictest29473-21diskmemload09:16:193
430899221,19cyclictest0-21swapper/309:35:133
430899220,22cyclictest0-21swapper/310:13:283
430899220,21cyclictest0-21swapper/312:05:273
430899220,1cyclictest0-21swapper/310:41:263
430899220,1cyclictest0-21swapper/309:55:043
430899220,1cyclictest0-21swapper/309:45:543
430899220,17cyclictest225-21systemd-journal10:34:593
430599220,22cyclictest0-21swapper/212:37:432
430599220,19cyclictest0-21swapper/211:40:422
430399220,20cyclictest0-21swapper/110:20:171
4308992110,10cyclictest29473-21diskmemload09:29:573
4308992110,10cyclictest29473-21diskmemload09:29:563
430899210,3cyclictest0-21swapper/309:11:403
430899210,20cyclictest4978-21rm11:10:163
430899210,1cyclictest0-21swapper/309:32:123
430899210,18cyclictest0-21swapper/311:30:143
4303992118,3cyclictest25-21ksoftirqd/112:38:101
430899209,9cyclictest9531-21ssh11:51:233
430899201,17cyclictest29473-21diskmemload11:56:453
430899201,16cyclictest29473-21diskmemload10:00:543
430899200,3cyclictest29473-21diskmemload10:26:213
430899200,1cyclictest0-21swapper/309:53:013
430899200,1cyclictest0-21swapper/309:23:523
430899200,16cyclictest0-21swapper/308:45:143
430599200,20cyclictest0-21swapper/209:27:482
430599200,20cyclictest0-21swapper/209:27:472
430599200,19cyclictest0-21swapper/210:45:272
430599200,19cyclictest0-21swapper/210:45:262
4303992018,2cyclictest23354-21kworker/1:011:01:551
430399200,18cyclictest0-21swapper/110:55:151
430899191,17cyclictest29473-21diskmemload11:16:113
430599190,19cyclictest0-21swapper/211:58:512
430399190,1cyclictest0-21swapper/111:07:481
430899181,16cyclictest29473-21diskmemload10:16:153
430899180,1cyclictest0-21swapper/312:26:263
430599180,18cyclictest0-21swapper/209:47:562
430599180,17cyclictest0-21swapper/211:49:222
430399184,7cyclictest16463-21ssh09:20:311
429899182,2cyclictest0-21swapper/010:45:370
429899182,2cyclictest0-21swapper/010:45:360
429899180,2cyclictest0-21swapper/011:57:540
429899180,2cyclictest0-21swapper/010:36:370
430899178,7cyclictest11345-21taskset07:24:123
430899171,15cyclictest7513-21fschecks_count07:15:173
430899171,14cyclictest28884-21cstates08:00:143
430899170,3cyclictest25481-21sed07:50:263
430899170,16cyclictest0-21swapper/308:20:133
430599173,2cyclictest141rcu_preempt09:31:412
430599170,17cyclictest0-21swapper/211:10:572
430399170,4cyclictest29473-21diskmemload12:01:011
4298991717,0cyclictest0-21swapper/010:58:130
430899160,15cyclictest8576-21expr08:25:153
430899160,15cyclictest13557-21chrt07:27:103
430899160,14cyclictest0-21swapper/309:05:003
430899160,13cyclictest0-21swapper/309:05:253
430599162,4cyclictest1142-21runrttasks11:50:092
430599160,16cyclictest0-21swapper/210:12:582
430599160,16cyclictest0-21swapper/209:58:022
430599160,15cyclictest0-21swapper/210:37:402
430599160,14cyclictest0-21swapper/210:56:252
429899162,13cyclictest0-21swapper/012:30:130
429899160,14cyclictest0-21swapper/009:40:220
430899150,14cyclictest18306-21chrt07:37:083
430899150,14cyclictest0-21swapper/308:35:113
430899150,10cyclictest0-21swapper/307:40:143
430599154,4cyclictest27338-21tr10:25:192
4305991512,1cyclictest0-21swapper/211:24:142
430599150,2cyclictest0-21swapper/211:08:532
430599150,15cyclictest0-21swapper/212:28:532
430399159,6cyclictest0-21swapper/109:48:101
430399159,5cyclictest21339-21cut07:45:141
4303991510,3cyclictest0-21swapper/110:52:291
4298991512,2cyclictest7145-21ssh12:29:380
429899150,1cyclictest0-21swapper/011:18:000
430899148,5cyclictest5655-21grep07:10:213
430899148,5cyclictest23426-21cat08:55:193
430899148,5cyclictest21006-21hwlatdetect08:50:183
430899148,5cyclictest16047-21taskset07:33:383
430899148,5cyclictest16047-21taskset07:33:373
430599142,11cyclictest8294-21ssh12:30:122
4305991410,1cyclictest225-21systemd-journal11:04:522
430599140,1cyclictest0-21swapper/212:08:452
430599140,14cyclictest0-21swapper/210:04:182
430399149,4cyclictest9036-21ls07:20:011
430399145,3cyclictest7839-21munin-run08:25:001
430399141,6cyclictest0-21swapper/110:40:131
4303991412,1cyclictest0-21swapper/111:31:241
4303991410,2cyclictest22009-21grep11:40:141
4298991413,1cyclictest0-21swapper/011:48:380
4298991410,3cyclictest30555-21cron09:30:010
4298991410,3cyclictest30555-21cron09:30:000
4298991410,2cyclictest8560-21tail07:15:260
429899140,2cyclictest0-21swapper/009:11:020
430899137,5cyclictest28220-21seq07:59:333
430899137,5cyclictest2408-21cat08:10:283
4308991310,2cyclictest615-21dump-pmu-power08:10:013
430899130,9cyclictest225-21systemd-journal07:50:013
430899130,1cyclictest0-21swapper/308:30:153
430599139,2cyclictest7650-21tail07:15:172
430599134,9cyclictest0-21swapper/210:09:482
430599132,8cyclictest13565-21diskmemload11:35:122
430599132,2cyclictest141rcu_preempt12:14:262
4305991312,1cyclictest4289-21cyclictest10:41:122
4305991310,2cyclictest7755-21sort09:35:172
4305991310,2cyclictest7546-21munin-run09:55:012
4305991310,2cyclictest6641-21turbostat07:14:592
4305991310,2cyclictest32043-21uname08:05:192
4305991310,2cyclictest29814-21wc08:00:222
4305991310,2cyclictest24276-21df07:50:152
4305991310,2cyclictest22835-21grep07:45:262
4305991310,2cyclictest17060-21sort07:35:162
4305991310,2cyclictest16022-21grep09:40:162
4305991310,2cyclictest15266-21grep09:20:192
430599130,2cyclictest0-21swapper/211:28:522
430599130,12cyclictest0-21swapper/211:17:482
430399139,3cyclictest31882-21grep09:10:261
430399139,3cyclictest27706-21grep10:45:161
430399139,3cyclictest27706-21grep10:45:151
430399139,2cyclictest24899-21cut07:50:201
430399138,4cyclictest11687-21sed10:35:201
4303991310,2cyclictest8105-21sed12:10:171
4303991310,2cyclictest6202-21grep07:10:281
4303991310,2cyclictest606-21sh08:10:001
4303991310,2cyclictest32309-21grep09:30:211
4303991310,2cyclictest30304-21grep11:45:151
4303991310,2cyclictest29588-21chrt12:23:251
4303991310,2cyclictest21307-21head08:50:201
4303991310,2cyclictest16122-21sh12:35:011
430399130,2cyclictest0-21swapper/111:20:511
430399130,13cyclictest0-21swapper/108:55:211
430399130,12cyclictest0-21swapper/110:02:181
429899139,3cyclictest14030-21chrt07:30:120
429899139,3cyclictest14030-21chrt07:30:120
429899137,2cyclictest3215-21apache_accesses10:50:130
4298991310,2cyclictest8082-21sort11:50:260
4298991310,2cyclictest31831-21ls12:25:010
4298991310,2cyclictest2379-21grep10:30:180
4298991310,2cyclictest22289-21grep07:45:190
4298991310,2cyclictest18430-21sort08:45:160
429899130,5cyclictest0-21swapper/009:58:300
429899130,12cyclictest15135-21sh09:20:180
430899125,6cyclictest3698-21expr08:15:143
430899120,11cyclictest0-21swapper/308:40:133
430599129,2cyclictest4087-21tail10:30:302
430599129,2cyclictest19023-21ssh10:20:212
430599129,2cyclictest17081-21grep08:40:252
430599129,2cyclictest11119-21sleep10:55:002
430599128,3cyclictest27193-21wc09:05:012
430599128,3cyclictest17968-21grep08:45:142
430599125,3cyclictest225-21systemd-journal09:09:592
430599123,8cyclictest7842-21idleruntime-cro08:25:002
430599120,12cyclictest0-21swapper/212:21:402
430599120,12cyclictest0-21swapper/209:14:242
430599120,11cyclictest5437-21sh08:19:592
430399129,2cyclictest8331-21grep09:55:141
430399129,2cyclictest32599-21ssh09:50:141
430399129,2cyclictest28360-21cat08:00:001
430399129,2cyclictest23360-21lockfile-create12:20:001
430399129,2cyclictest18596-21head08:45:171
430399129,2cyclictest18136-21head07:35:271
430399129,2cyclictest17394-21head09:40:281
430399129,2cyclictest15770-21head07:30:271
430399129,2cyclictest15770-21head07:30:261
4303991211,1cyclictest0-21swapper/112:25:481
4303991210,1cyclictest0-21swapper/109:05:131
429899129,2cyclictest26878-21head07:55:170
429899129,2cyclictest22873-21wc08:55:150
429899129,2cyclictest19507-21sort07:40:180
429899129,2cyclictest19093-21tr10:40:150
429899129,2cyclictest17170-21grep12:35:160
429899129,2cyclictest16444-21if_enp0s31f612:15:170
429899129,2cyclictest16225-21sed08:40:160
429899129,2cyclictest14918-21cut11:35:220
429899129,2cyclictest10753-21cat07:20:260
429899121,6cyclictest0-21swapper/010:19:100
429899121,10cyclictest21086-21grep11:20:160
4298991210,1cyclictest10697-21chrt08:30:130
429899120,7cyclictest29479-21idleruntime-cro09:09:590
429899120,12cyclictest0-21swapper/009:31:410
429899120,11cyclictest1980-21sed10:10:170
429899120,0cyclictest0-21swapper/010:25:110
430599119,1cyclictest27846-21head07:55:272
430599118,2cyclictest20337-21cut08:50:152
430599118,2cyclictest15416-21which07:30:242
430599118,2cyclictest15416-21which07:30:232
430599118,2cyclictest13096-21cut08:35:142
430599118,2cyclictest12908-21head07:25:202
430599118,2cyclictest11396-21tail10:15:272
430599117,3cyclictest9422-21wc08:25:232
430599117,3cyclictest15387-21ssh12:15:082
430399119,1cyclictest9030-21ssh11:50:591
430399118,2cyclictest8138-21grep08:25:141
430399118,2cyclictest6153-21cut09:15:141
430399118,2cyclictest5044-21grep08:15:251
430399118,2cyclictest32125-21head12:05:171
430399118,2cyclictest29482-21sh11:25:131
430399118,2cyclictest28837-21grep08:00:151
430399118,2cyclictest25727-21grep09:00:191
430399118,2cyclictest22821-21aten2_r7power_p09:25:141
430399118,2cyclictest22821-21aten2_r7power_p09:25:131
430399118,2cyclictest1795-21sh10:30:021
430399118,2cyclictest1607-21grep08:10:181
430399118,2cyclictest15726-21tr08:40:131
430399118,2cyclictest15547-21basename11:55:181
430399118,2cyclictest14743-21grep08:35:261
430399118,2cyclictest12047-21expr07:25:131
430399118,2cyclictest11326-21taskset07:23:581
430399117,3cyclictest4783-21sort11:10:171
430399113,5cyclictest1741-21ssh10:10:151
430399110,10cyclictest11048-21rm09:37:001
429899118,2cyclictest9252-21sh09:16:260
429899118,2cyclictest9206-21strings08:25:210
429899118,2cyclictest5210-21sort07:10:180
429899118,2cyclictest4033-21grep08:15:160
429899118,2cyclictest31714-21rm10:09:120
429899118,2cyclictest31165-21ssh12:05:080
429899118,2cyclictest25888-21cron07:55:010
429899118,2cyclictest23882-21basename12:00:180
429899118,2cyclictest23595-21ssh09:45:100
429899118,2cyclictest19718-21ssh11:00:050
429899118,2cyclictest19546-21sh10:01:090
429899118,2cyclictest18882-21cat10:20:210
429899118,2cyclictest13319-21tr08:35:140
429899118,2cyclictest10646-21ssh09:36:430
429899118,2cyclictest10288-21ssh12:11:170
429899117,3cyclictest29727-21sh11:25:130
429899117,3cyclictest12016-21ls11:15:010
429899111,2cyclictest0-21swapper/011:40:140
429899110,4cyclictest0-21swapper/011:34:260
430599108,1cyclictest2732-21taskset08:11:472
430599108,1cyclictest243632sleep208:56:012
430599108,1cyclictest19487-21cut07:40:172
430599107,2cyclictest10986-21wc08:30:152
430599107,2cyclictest10184-21if_enp2s007:20:182
430399107,2cyclictest19258-21expr07:40:151
430399107,2cyclictest10646-21chrt08:30:021
429899108,1cyclictest7211-21ls08:20:260
429899108,1cyclictest28962-21ls08:00:150
429899108,1cyclictest20610-21cat08:50:160
429899108,1cyclictest13181-21cat07:25:240
429899107,2cyclictest31403-21expr08:05:140
429899107,2cyclictest25761-21hddtemp_smartct09:00:190
429899107,2cyclictest1839-21ssh09:50:350
429899107,2cyclictest18051-21grep07:35:260
429899107,2cyclictest1071-21ls08:10:160
429899106,3cyclictest28150-21ssh11:05:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional