You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-23 - 22:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sat May 23, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
210512173102,28sleep20-21swapper/207:08:272
211122164110,28sleep10-21swapper/107:09:151
211192163109,28sleep30-21swapper/307:09:213
209252160101,30sleep00-21swapper/007:06:500
2915621340,2sleep00-21swapper/012:38:350
258192680,4sleep025810-21perf08:35:000
258192680,4sleep025810-21perf08:34:590
52652600,2sleep00-21swapper/009:30:300
268682550,2sleep10-21swapper/109:23:051
268682550,2sleep10-21swapper/109:23:041
91102520,1sleep20-21swapper/211:59:202
304882510,2sleep00-21swapper/009:25:240
296702510,1sleep00-21swapper/008:40:250
43322490,2sleep30-21swapper/310:18:293
215592460,1sleep30-21swapper/312:08:193
21445994613,23cyclictest5547-21sh11:31:472
326102450,1sleep30-21swapper/309:27:083
129832450,1chrt12984-21/usr/sbin/munin10:00:251
2144599441,31cyclictest29301-21/usr/sbin/munin07:25:242
2144599441,26cyclictest8392-21diskmemload09:25:342
21445994411,23cyclictest24512-21sh09:20:422
21445994411,23cyclictest24512-21sh09:20:412
147832440,1sleep20-21swapper/210:01:542
69122430,1sleep20-21swapper/210:44:562
2144599431,26cyclictest8392-21diskmemload11:11:172
2144599431,25cyclictest8392-21diskmemload09:14:242
2144599430,26cyclictest8392-21diskmemload09:37:232
2144599421,27cyclictest8392-21diskmemload10:36:342
2144599421,27cyclictest11966-21rm11:36:312
2144599421,26cyclictest8392-21diskmemload10:25:532
2144599421,25cyclictest8392-21diskmemload11:16:382
21445994211,17cyclictest2833-21sh09:53:212
21445994210,23cyclictest8392-21diskmemload09:43:052
2144599420,26cyclictest8392-21diskmemload12:34:472
2144399427,25cyclictest24175-21cat12:35:010
2144599419,17cyclictest26563-21rm11:23:402
2144599411,29cyclictest8655-21sh09:57:302
2144599411,29cyclictest25038-21sh11:46:032
2144599411,25cyclictest8392-21diskmemload12:00:292
2144599401,29cyclictest21672-21sh10:30:442
2144599401,27cyclictest8392-21diskmemload12:29:342
2144599401,25cyclictest8392-21diskmemload09:45:532
2144599401,23cyclictest18486-21ssh11:41:232
21445994012,14cyclictest30344-21sh11:01:392
2144599400,21cyclictest3290-21ssh09:30:012
2144399409,19cyclictest30942-21seq11:02:160
2144399408,17cyclictest28322-21ssh10:35:310
21443994014,10cyclictest2696-21/usr/sbin/munin11:05:200
2144599391,28cyclictest31204-21sh10:14:502
2144599391,28cyclictest23149-21sh10:08:242
2144599391,27cyclictest8994-21ssh10:21:252
2144599391,27cyclictest8024-21sh12:22:202
2144599391,27cyclictest22676-21rm10:55:392
2144599391,20cyclictest27084-21sh12:36:242
2144599391,19cyclictest19961-21ssh10:54:402
2144599391,19cyclictest19961-21ssh10:54:402
2144399399,14cyclictest20014-21ssh11:42:560
2144599386,18cyclictest16816-21sh12:05:032
2144599384,26cyclictest8392-21diskmemload12:12:072
2144599381,27cyclictest4188-21sh11:06:162
2144599381,19cyclictest8392-21diskmemload11:53:542
2144599380,28cyclictest2132-21sh12:18:052
2144599380,24cyclictest3795-21chrt08:56:212
2144399389,12cyclictest11083-21ssh09:10:570
2144399388,16cyclictest13015-21ssh12:01:230
21443993811,10cyclictest8392-21diskmemload10:50:020
21443993811,10cyclictest8392-21diskmemload10:50:020
21443993810,9cyclictest8392-21diskmemload12:29:190
21443993810,9cyclictest8392-21diskmemload10:13:410
21443993810,9cyclictest20301-21taskset09:18:140
21443993810,17cyclictest8392-21diskmemload10:47:490
2144599370,27cyclictest19031-21sh09:16:582
2144599370,21cyclictest9345-21ssh10:45:372
2144599370,20cyclictest31841-21chrt07:31:022
2144599370,20cyclictest2028-21ssh11:29:562
2144599370,20cyclictest18993-21taskset08:15:322
2144399379,16cyclictest9528-21ssh10:21:570
2144399379,15cyclictest2982-21sh11:54:510
2144399379,10cyclictest22586-21ssh10:31:390
2144399378,12cyclictest24776-21ssh11:45:470
21443993711,9cyclictest8392-21diskmemload09:52:520
21443993711,9cyclictest20368-21ssh09:42:020
21443993710,10cyclictest8392-21diskmemload12:13:550
2144599360,20cyclictest6136-21chrt09:03:422
2144599360,19cyclictest23181-21chrt07:12:452
2144599360,19cyclictest1653-21sh10:15:452
2144399369,16cyclictest8392-21diskmemload10:43:330
2144399368,9cyclictest30378-21ssh11:25:480
2144399368,8cyclictest26391-21ssh11:23:290
2144399368,18cyclictest7139-21ssh12:21:250
2144399368,15cyclictest2572-21ssh12:18:320
2144399368,15cyclictest16036-21ssh10:26:470
2144399368,14cyclictest3414-21ssh10:17:320
2144399368,11cyclictest8014-21ssh09:56:490
2144399367,10cyclictest13265-21ssh11:37:500
21443993610,9cyclictest24012-21ssh10:09:150
2144599351,25cyclictest1893-21taskset08:54:232
2144599350,21cyclictest17023-21chrt08:12:422
2144399359,9cyclictest32152-21seq08:49:090
2144399359,16cyclictest26028-21sh09:22:140
2144399359,16cyclictest26028-21sh09:22:130
2144399358,9cyclictest11939-21ssh09:35:300
2144399358,10cyclictest3813-21taskset07:40:510
2144399357,17cyclictest10467-21ssh11:10:520
21443993511,9cyclictest8392-21diskmemload11:15:480
21443993510,11cyclictest22065-21python310:55:230
21443993510,11cyclictest19182-21ssh12:05:540
21443993510,10cyclictest8392-21diskmemload09:45:310
2144699343,11cyclictest8858-21ssh09:10:143
2144599341,19cyclictest27896-21chrt08:39:402
2144399349,9cyclictest8392-21diskmemload11:34:480
21443993411,8cyclictest13529-21ssh10:00:370
101192340,2chrt0-21swapper/112:24:271
2144599331,18cyclictest27752-21taskset07:23:282
2144599330,20cyclictest8044-21chrt09:05:442
2144599330,20cyclictest30020-21runrttasks08:44:102
2144399339,8cyclictest8392-21diskmemload11:55:400
21445993214,16cyclictest23617-21find07:15:102
21445993213,16cyclictest22119-21entropy08:25:162
2144599321,18cyclictest14737-21runrttasks08:05:472
2144399326,9cyclictest26003-21dpkg07:20:120
21445993112,16cyclictest24119-21cat08:30:142
21445993112,16cyclictest24119-21cat08:30:142
2144599311,19cyclictest10577-21runrttasks07:57:372
2144599311,18cyclictest8374-21taskset07:51:482
2144599310,18cyclictest3901-21taskset07:42:022
2144399308,9cyclictest636-21runrttasks07:25:550
2144399298,8cyclictest636-21runrttasks08:25:320
2144399287,8cyclictest25338-21taskset07:17:540
2144399286,8cyclictest0-21swapper/009:09:270
2144399286,8cyclictest0-21swapper/007:47:250
21443992819,4cyclictest4363-21apt-get09:00:120
2144599277,17cyclictest11312-21expr08:00:132
2144399276,7cyclictest1875-21taskset07:38:450
170652270,3sleep30-21swapper/309:40:123
21446992614,8cyclictest1643-21memory10:40:193
2144599261,22cyclictest987-21/usr/sbin/munin07:35:362
2144599261,22cyclictest5500-21run-parts07:45:242
2144599261,22cyclictest30349-21/usr/sbin/munin08:45:122
2144399252,8cyclictest12676-21runrttasks08:01:500
21443992513,9cyclictest16770-21grep08:10:270
21443992513,9cyclictest0-21swapper/007:10:010
21443992511,12cyclictest3647-21tail08:55:270
21443992511,11cyclictest26386-21df_inode08:35:160
227412240,2sleep122-21ksoftirqd/110:07:591
2144599241,21cyclictest21013-21unixbench-2d08:20:232
2144399243,7cyclictest0-21swapper/007:30:240
21443992415,7cyclictest17608-21cpuspeed_turbos08:15:130
21443992414,8cyclictest20067-21grep08:20:140
21443992412,10cyclictest14698-21chrt08:05:270
82432230,3sleep30-21swapper/312:22:333
288972230,10sleep122-21ksoftirqd/107:25:191
21446992315,5cyclictest12568-21sh10:48:533
2144499237,4cyclictest3104-21grep11:30:171
21444992317,4cyclictest22-21ksoftirqd/109:45:011
21444992316,5cyclictest22-21ksoftirqd/110:40:001
21444992316,5cyclictest15069-21cron08:10:001
2144499231,5cyclictest0-21swapper/109:45:511
21443992314,6cyclictest32485-21apt-get08:50:120
21443992312,8cyclictest9402-21gltestperf07:55:170
2144699222,13cyclictest4939-21ssh09:55:133
21446992214,5cyclictest636-21runrttasks10:02:043
21446992213,6cyclictest278-21systemd-journal12:20:003
21446992211,8cyclictest12043-21awk08:00:233
2144499222,15cyclictest8392-21diskmemload10:30:121
21444992216,4cyclictest22-21ksoftirqd/111:10:011
21444992216,4cyclictest22-21ksoftirqd/110:20:011
21444992216,4cyclictest22-21ksoftirqd/109:00:001
2144499221,4cyclictest0-21swapper/112:19:341
21443992212,7cyclictest7595-21grep07:50:190
22652210,1sleep10-21swapper/109:52:461
2144699218,10cyclictest0-21swapper/311:20:233
2144699213,5cyclictest28927-21sh11:50:003
2144699213,12cyclictest28276-21apt-get11:25:133
21446992113,5cyclictest8392-21diskmemload12:35:163
21446992112,6cyclictest4478-21missed_timers11:55:203
21446992111,6cyclictest15494-21sed11:40:003
2144499214,8cyclictest9267-21ssh11:35:141
21444992113,6cyclictest25764-21seq08:34:441
21444992113,6cyclictest25764-21seq08:34:431
21444992112,7cyclictest560-21dbus-daemon08:19:591
21444992112,7cyclictest31120-21ls08:45:181
37252200,3sleep334-21ksoftirqd/309:30:133
2144699203,11cyclictest10344-21apt-get12:00:133
2144699202,12cyclictest9539-21ntp_states11:10:323
21446992012,6cyclictest30826-21cat11:50:243
21446992012,6cyclictest13628-21tune2fs08:05:163
21446992012,5cyclictest19357-21basename08:19:593
21446992011,6cyclictest920-21ntpq07:35:343
2144499202,12cyclictest28212-21apt-get08:40:131
21444992015,4cyclictest22-21ksoftirqd/108:30:001
21444992014,5cyclictest22-21ksoftirqd/109:15:001
21444992014,4cyclictest22-21ksoftirqd/112:00:001
21444992014,4cyclictest22-21ksoftirqd/107:20:011
21444992014,4cyclictest15180-21sh12:27:511
21444992012,6cyclictest278-21systemd-journal07:35:001
21444992012,6cyclictest27557-21seq07:20:571
21444992012,6cyclictest26980-21ls11:00:001
21444992012,6cyclictest23033-21seq07:10:501
21444992011,7cyclictest23900-21seq11:20:571
2144699199,7cyclictest29070-21awk07:25:223
2144699192,10cyclictest6327-21ssh10:20:133
2144699191,8cyclictest31394-21latency_hist10:15:013
2144699191,8cyclictest1629-21apt-get11:05:113
21446991912,4cyclictest8392-21diskmemload09:19:223
21446991912,4cyclictest20295-21cut10:30:173
21446991912,4cyclictest14976-21sh10:50:203
21446991912,4cyclictest14976-21sh10:50:193
21446991911,6cyclictest28547-21df08:40:153
21446991911,5cyclictest3895-21switchtime11:30:243
21446991911,5cyclictest10360-21grep07:55:263
2144699191,13cyclictest11079-21apt-get12:25:123
21446991910,6cyclictest26077-21sh12:11:113
2144699190,16cyclictest30814-21gltestperf08:45:163
2144499192,12cyclictest18762-21ntp_states12:30:211
2144499191,8cyclictest9555-21ntp_states11:10:321
21444991914,3cyclictest4202-21kernelversion09:30:171
21444991913,4cyclictest5596-21sh10:43:361
21444991913,4cyclictest23803-21open_inodes11:45:211
21444991912,5cyclictest8872-21find07:55:111
21444991912,5cyclictest5354-21pmu-power07:45:221
21444991912,5cyclictest17159-21seq08:14:301
21444991912,4cyclictest6127-21sh10:20:021
21444991911,6cyclictest769-21cut07:35:321
21444991911,6cyclictest6834-21seq09:05:141
21444991911,6cyclictest4565-21expr09:00:131
21444991911,6cyclictest24986-21cat12:10:221
21444991911,6cyclictest21021-21grep08:20:241
21444991911,6cyclictest16715-21seq10:27:291
21444991911,6cyclictest14806-21seq09:38:241
21444991911,6cyclictest10868-21fw_packets12:00:161
21444991911,5cyclictest560-21dbus-daemon10:10:061
21444991910,7cyclictest17191-21date09:15:241
21444991910,7cyclictest17182-21cat12:05:121
21444991910,6cyclictest844-21cat08:50:191
2144699182,12cyclictest28907-21ssh11:00:243
2144699182,11cyclictest30649-21apt-get09:50:143
2144699182,11cyclictest19626-21apt-get08:20:123
2144699182,11cyclictest11225-21ntp_states09:35:223
21446991812,4cyclictest24862-21sh09:21:043
21446991812,4cyclictest24862-21sh09:21:043
21446991811,5cyclictest7206-21cut07:50:163
21446991811,5cyclictest17984-21expr12:30:143
21446991811,4cyclictest8411-21sh09:09:593
21446991810,6cyclictest31631-21sed07:30:233
21446991810,6cyclictest2155-21sed08:55:013
21446991810,6cyclictest19381-21grep10:30:013
21446991810,5cyclictest52950irq/141-enp3s0-10:55:213
21446991810,5cyclictest24618-21cat08:30:183
21446991810,5cyclictest24618-21cat08:30:173
2144499189,6cyclictest14502-21cat10:50:161
2144499189,6cyclictest14502-21cat10:50:161
2144499187,8cyclictest27457-21grep08:35:261
2144499182,11cyclictest24376-21apt-get12:35:131
21444991813,3cyclictest16575-21munin-node11:40:111
21444991812,4cyclictest30396-21sh11:01:421
21444991812,4cyclictest28960-21sh11:25:171
21444991812,4cyclictest14946-21/usr/sbin/munin11:15:251
21444991812,4cyclictest10325-21sh09:59:101
21444991811,5cyclictest12636-21seq08:01:181
21444991811,4cyclictest8247-21latency10:45:191
21444991811,4cyclictest516-21sh11:52:221
21444991810,6cyclictest29883-21basename09:25:171
2144699179,6cyclictest23039-21cat08:25:243
2144699179,6cyclictest15580-21cut11:15:163
2144699177,8cyclictest0-21swapper/311:40:473
2144699172,12cyclictest3281-21wc07:40:223
2144699171,8cyclictest640-21snmpd09:02:573
21446991711,4cyclictest22086-21sh10:07:193
21446991711,4cyclictest17201-21cron08:15:003
2144699171,13cyclictest26639-21tr10:35:133
21446991710,5cyclictest27332-21wc08:35:263
21446991710,5cyclictest26285-21expr07:20:143
21446991710,5cyclictest24207-21expr09:45:133
21446991710,5cyclictest22186-21sed07:10:173
21444991711,4cyclictest7671-21sh07:50:211
21444991710,5cyclictest3409-21grep07:40:241
2144699169,5cyclictest24529-21ntpq07:15:203
2144699168,6cyclictest2270-21cat08:55:133
21446991611,3cyclictest4375-21/usr/sbin/munin07:45:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional