You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-07 - 15:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Mon Sep 07, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
162592179104,27sleep00-21swapper/007:09:200
159722168104,28sleep10-21swapper/107:05:361
161172161103,29sleep20-21swapper/207:07:302
160142161103,29sleep30-21swapper/307:06:103
161292710,1sleep30-21swapper/309:42:063
154882690,2sleep00-21swapper/011:18:240
42592620,4sleep24260-21ls11:10:142
243072600,2sleep2271rcuc/211:00:232
259272550,1sleep325928-21if_enp3s011:50:173
101272480,2chrt0-21swapper/311:14:463
97902470,1sleep10-21swapper/111:38:221
16783994713,19cyclictest26881-21sh11:26:232
16783994713,17cyclictest6065-21sh12:00:052
104152470,1sleep00-21swapper/012:02:570
245202460,1sleep10-21swapper/109:48:471
16783994613,17cyclictest15237-21sh10:54:242
16783994610,26cyclictest5661-21sh11:59:562
16783994513,16cyclictest26547-21sh12:39:172
16783994513,15cyclictest8923-21sh10:49:502
123852450,1sleep20-21swapper/211:40:162
274782440,1sleep10-21swapper/111:26:581
16783994413,16cyclictest10693-21sh12:27:002
16783994410,16cyclictest19542-21sh12:33:552
1678399436,20cyclictest9180-21sh09:13:042
1678399435,21cyclictest1975-21sh09:31:232
1678399435,21cyclictest1975-21sh09:31:232
1678399431,27cyclictest2642-21kworker/u8:010:07:362
1678399426,19cyclictest13969-21sh11:16:532
16783994210,18cyclictest32140-21sh09:54:422
204842410,1sleep00-21swapper/009:20:530
179942410,2sleep00-21swapper/007:10:240
179942410,2sleep00-21swapper/007:10:230
16783994110,15cyclictest27934-21sh12:15:242
1678399394,21cyclictest4375-21diskmemload10:33:222
1678399389,15cyclictest32364-21sh10:18:452
1678399389,15cyclictest32364-21sh10:18:442
52002370,2sleep30-21swapper/311:59:273
1678399371,22cyclictest0-21swapper/211:21:282
1678399364,18cyclictest4375-21diskmemload10:35:412
16783993521,12cyclictest0-21swapper/207:42:212
1678399341,21cyclictest1797-21kworker/u8:212:22:422
257002330,2chrt0-21swapper/111:01:271
1678399331,7cyclictest17803-21ssh09:19:582
1678399330,7cyclictest32670-21ssh09:55:022
1678399326,23cyclictest18473-21rm11:45:082
16783993214,15cyclictest20557-21date07:15:252
162632320,6sleep0131rcuc/011:43:070
115862320,2sleep10-21swapper/109:39:201
26822310,1chrt0-21swapper/009:55:570
1678399318,21cyclictest3115-21sed07:50:252
1678399318,20cyclictest6098-21grep09:35:142
1678399318,20cyclictest18616-21expr09:20:142
1678399317,21cyclictest30113-21expr11:05:152
1678399316,22cyclictest22242-21cat08:35:172
1678399315,23cyclictest26755-21cat11:50:242
1678399312,24cyclictest0-21swapper/208:10:112
1678399311,5cyclictest7793-21sshd10:00:172
16783993113,15cyclictest27199-21seq07:31:112
16783993113,15cyclictest17111-21sh10:55:182
1678399310,8cyclictest4981-21/usr/sbin/munin07:55:212
1678399310,4cyclictest230102chrt07:22:182
1678399307,20cyclictest7076-21latency08:00:172
1678399307,20cyclictest7076-21latency08:00:162
1678399302,3cyclictest0-21swapper/208:05:122
1678399301,23cyclictest30878-21ssh11:30:112
16783993010,17cyclictest24045-21which07:25:162
16783993010,17cyclictest21146-21seq08:32:292
1678399300,6cyclictest25281-21cstates09:25:132
1678399300,3cyclictest4375-21diskmemload10:40:412
1678399300,3cyclictest31494-21systemctl08:55:212
1678399300,2cyclictest24160-21ssh09:48:262
1678399300,28cyclictest0-21swapper/207:49:412
1678399299,18cyclictest28415-21cat08:50:152
1678399298,18cyclictest816-21gltestperf09:00:162
1678399298,18cyclictest4519-21latency_hist09:10:002
1678399296,20cyclictest27584-21grep08:45:252
1678399296,20cyclictest27584-21grep08:45:242
1678399292,24cyclictest13136-21grep08:15:142
1678399291,5cyclictest14389-21grep12:05:262
1678399291,25cyclictest14693-21basename10:30:002
1678399290,7cyclictest6109-21ssh10:23:092
1678399290,5cyclictest20623-21ssh10:10:132
1678399290,3cyclictest4375-21diskmemload12:11:252
1678399290,2cyclictest188952sleep208:26:022
16780992921,6cyclictest27266-21cron12:40:001
1677499291,26cyclictest12974-21packagekitd08:19:130
128932290,2sleep30-21swapper/310:28:133
1678399288,17cyclictest24572-21basename08:40:172
1678399285,20cyclictest13100-21forks09:40:142
1678099281,25cyclictest0-21swapper/108:47:361
1678099281,25cyclictest0-21swapper/108:47:351
1678099281,25cyclictest0-21swapper/108:09:161
1678099281,25cyclictest0-21swapper/107:55:361
1677499281,25cyclictest0-21swapper/007:47:570
1677499281,25cyclictest0-21swapper/007:41:590
1677499281,25cyclictest0-21swapper/007:32:410
163262280,19sleep30-21swapper/312:07:093
243652270,3sleep10-21swapper/109:24:471
1678599271,24cyclictest0-21swapper/307:48:223
1678599271,24cyclictest0-21swapper/307:27:563
1678399275,20cyclictest648-21ntpd11:35:172
16783992725,1cyclictest0-21swapper/208:21:452
1678399270,6cyclictest18853-21cat07:15:012
1678399270,6cyclictest18853-21cat07:15:002
16780992724,2cyclictest0-21swapper/108:22:031
1678099271,2cyclictest0-21swapper/108:13:441
1678099271,24cyclictest0-21swapper/108:41:411
1678099271,24cyclictest0-21swapper/108:37:251
1678099271,24cyclictest0-21swapper/108:32:481
1678099271,24cyclictest0-21swapper/108:17:261
1678099271,24cyclictest0-21swapper/107:46:211
1678099271,24cyclictest0-21swapper/107:40:351
1678099271,24cyclictest0-21swapper/107:25:061
1678099271,24cyclictest0-21swapper/107:22:271
1677499271,2cyclictest0-21swapper/008:27:110
1677499271,25cyclictest0-21swapper/008:21:350
1677499271,25cyclictest0-21swapper/008:14:150
1677499271,24cyclictest0-21swapper/008:51:560
1677499271,24cyclictest0-21swapper/008:46:040
1677499271,24cyclictest0-21swapper/008:46:030
1677499271,24cyclictest0-21swapper/008:41:450
1677499271,24cyclictest0-21swapper/008:31:440
1677499271,24cyclictest0-21swapper/008:09:140
1677499271,24cyclictest0-21swapper/007:56:010
1677499271,24cyclictest0-21swapper/007:50:290
1677499271,24cyclictest0-21swapper/007:35:510
1677499271,24cyclictest0-21swapper/007:26:300
1677499271,24cyclictest0-21swapper/007:22:120
291332260,8sleep012-21ksoftirqd/009:27:510
1678599261,24cyclictest0-21swapper/308:54:153
1678599261,23cyclictest0-21swapper/308:39:363
1678599261,23cyclictest0-21swapper/308:21:313
1678599261,23cyclictest0-21swapper/308:16:413
1678599261,23cyclictest0-21swapper/308:09:163
1678599261,23cyclictest0-21swapper/308:04:503
1678599261,23cyclictest0-21swapper/308:04:503
1678599261,23cyclictest0-21swapper/307:55:223
1678599261,23cyclictest0-21swapper/307:50:363
1678599261,23cyclictest0-21swapper/307:36:433
1678599261,23cyclictest0-21swapper/307:20:273
1678399265,18cyclictest0-21swapper/207:35:152
16780992624,1cyclictest0-21swapper/107:36:181
16780992623,2cyclictest0-21swapper/107:51:201
1678099261,24cyclictest0-21swapper/108:26:271
1678099261,1cyclictest0-21swapper/107:19:181
1678099260,24cyclictest0-21swapper/108:50:551
1678099260,24cyclictest0-21swapper/108:05:001
1678099260,24cyclictest0-21swapper/108:04:591
1678099260,24cyclictest0-21swapper/107:30:061
16774992618,6cyclictest24362-21ssh12:37:030
1677499260,24cyclictest0-21swapper/007:19:440
103832260,10sleep334-21ksoftirqd/312:26:403
207262250,2sleep30-21swapper/310:58:113
16785992523,1cyclictest0-21swapper/308:26:313
1678599251,23cyclictest0-21swapper/308:46:293
1678599251,23cyclictest0-21swapper/308:46:293
1678599251,23cyclictest0-21swapper/307:41:003
1678599251,23cyclictest0-21swapper/307:30:523
1678599251,22cyclictest0-21swapper/309:09:463
1678599250,23cyclictest0-21swapper/308:30:573
16780992523,1cyclictest0-21swapper/108:55:101
16780992515,8cyclictest278-21systemd-journal09:20:001
1678099251,1cyclictest0-21swapper/109:02:521
297672240,3sleep122-21ksoftirqd/110:16:081
297672240,3sleep122-21ksoftirqd/110:16:071
246802240,2sleep30-21swapper/311:49:523
226462240,1sleep10-21swapper/110:35:191
16785992422,1cyclictest0-21swapper/308:13:213
16785992422,1cyclictest0-21swapper/307:19:503
1678599241,22cyclictest0-21swapper/308:43:543
16780992413,9cyclictest26616-21fschecks_count09:50:171
1677499240,1cyclictest0-21swapper/009:09:490
1677499240,1cyclictest0-21swapper/008:56:110
1677499240,1cyclictest0-21swapper/008:03:590
1677499240,1cyclictest0-21swapper/008:03:590
283212230,2sleep10-21swapper/111:51:471
265922230,5sleep326598-21cpuspeed_turbos12:15:123
194922230,3sleep10-21swapper/111:45:201
18552230,1sleep10-21swapper/110:20:131
177972230,3sleep00-21swapper/010:55:230
176592230,2sleep3331rcuc/312:32:033
1678599230,1cyclictest0-21swapper/309:10:103
1678599230,1cyclictest0-21swapper/308:55:093
1678099232,15cyclictest16248-21ssh10:55:111
16780992313,7cyclictest15621-21fw_packets10:30:151
16774992315,6cyclictest29544-21ssh10:15:550
16774992315,6cyclictest29544-21ssh10:15:540
16774992314,6cyclictest13590-21grep09:40:190
1678599223,12cyclictest52950irq/141-enp3s0-11:05:133
16785992215,5cyclictest45-21kcompactd009:50:123
16785992214,5cyclictest12268-21sendmail09:40:013
16785992214,4cyclictest25747-21ls09:50:003
16780992213,7cyclictest13908-21ntp_kernel_pll_12:05:211
16780992213,6cyclictest14155-21sh12:30:041
16780992210,3cyclictest22-21ksoftirqd/112:23:021
1677499229,4cyclictest23651-21basename11:00:160
16774992220,1cyclictest0-21swapper/009:02:370
1677499221,19cyclictest24703-21grep11:25:160
295062210,1sleep00-21swapper/009:52:030
252762210,2sleep00-21swapper/010:37:090
16785992113,6cyclictest13390-21ls11:40:253
1678099212,4cyclictest0-21swapper/110:53:501
1678099212,14cyclictest4375-21diskmemload09:25:131
16780992111,7cyclictest11344-21gltestperf11:15:161
16780992110,8cyclictest3725-21basename11:10:001
16774992113,6cyclictest16739-21sed10:30:250
1678599202,13cyclictest12864-21ssh09:15:223
1678599201,7cyclictest8172-21kworker/u8:112:11:353
16785992014,4cyclictest26894-21idleruntime-cro10:14:593
16785992014,4cyclictest26298-21sendmail_mailtr09:25:223
16785992013,4cyclictest15823-21cron10:55:003
1678099209,7cyclictest52850irq/140-enp3s0-09:30:131
1678099209,7cyclictest52850irq/140-enp3s0-09:30:121
1678099202,10cyclictest19182-21apt-get12:10:041
16780992014,4cyclictest22-21ksoftirqd/112:05:011
16780992012,6cyclictest31891-21ntpq11:30:201
1678099201,13cyclictest7195-21ssh10:00:141
16780992010,8cyclictest636-21runrttasks10:49:101
16780992010,7cyclictest342-21grep11:55:191
16774992018,1cyclictest0-21swapper/008:35:060
1677499201,6cyclictest0-21swapper/011:05:240
16774992014,4cyclictest1946-21sh09:31:230
16774992014,4cyclictest1946-21sh09:31:220
16774992013,4cyclictest4375-21diskmemload10:20:190
16774992012,6cyclictest23447-21seq11:48:370
16774992012,6cyclictest20231-21sh11:21:230
16774992011,6cyclictest11904-21diskstats09:15:130
16774992010,6cyclictest52750irq/139-enp3s0-10:45:120
150772200,1sleep30-21swapper/310:30:113
1678599199,8cyclictest798-21tune2fs09:00:153
1678599197,8cyclictest18302-21apt-get09:20:133
1678599192,11cyclictest75-21kswapd010:00:123
16785991913,4cyclictest4229-21sh10:21:173
16785991912,4cyclictest6155-21/usr/sbin/munin12:00:203
16785991911,6cyclictest31694-21sed11:30:173
16785991911,6cyclictest24199-21apt-get11:25:113
16785991911,6cyclictest22968-21grep10:35:213
16785991911,6cyclictest18213-21unixbench-2d07:10:263
16785991911,6cyclictest18213-21unixbench-2d07:10:253
16785991910,7cyclictest11407-21cron11:39:593
1678099199,6cyclictest52850irq/140-enp3s0-09:55:111
1678099192,11cyclictest28577-21apt-get10:40:121
16780991912,5cyclictest278-21systemd-journal09:14:591
16780991912,5cyclictest18223-21seq09:44:131
16780991912,4cyclictest4375-21diskmemload11:42:051
1677499199,7cyclictest8496-21ntp_states12:25:210
1677499192,11cyclictest9626-21apt-get10:50:180
16774991914,3cyclictest24386-21sh10:12:300
16774991913,4cyclictest9902-21sh11:14:300
16774991913,4cyclictest9778-21sh11:38:220
16774991913,4cyclictest4499-21sh11:58:450
16774991913,4cyclictest32216-21switchtime11:30:220
16774991913,4cyclictest28424-21sh11:51:530
16774991911,6cyclictest9133-21seq09:13:030
16774991911,6cyclictest30570-21sh12:17:500
16774991911,6cyclictest14834-21fw_packets12:30:150
16774991910,7cyclictest20407-21seq12:10:150
1678599188,8cyclictest31907-21cstates09:30:143
1678599188,8cyclictest31907-21cstates09:30:133
1678599187,7cyclictest28193-21sed10:15:193
1678599187,7cyclictest28193-21sed10:15:183
16785991812,4cyclictest2304-21sh10:44:553
16785991811,5cyclictest15558-21awk10:05:253
16785991811,4cyclictest4375-21diskmemload11:00:083
16785991810,6cyclictest21245-21expr12:35:123
16785991810,6cyclictest17105-21systemctl11:20:003
1678099189,7cyclictest560-21dbus-daemon12:19:591
1678099188,7cyclictest20961-21df_inode10:10:161
1678099184,11cyclictest18976-21/usr/sbin/munin11:20:241
1678099183,9cyclictest13850-21apt-get10:05:111
1678099181,7cyclictest6309-21runrttasks11:10:531
16780991811,4cyclictest8344-21acpi10:25:111
16780991810,6cyclictest8310-21memory12:25:191
1677499189,7cyclictest30650-21seq10:40:470
1677499182,12cyclictest13868-21ssh10:05:130
16774991812,4cyclictest7984-21sh09:35:420
16774991811,5cyclictest9082-21seq10:00:370
16774991811,5cyclictest2005-21sendmail_mailqu12:20:230
16774991811,5cyclictest14866-21seq12:05:430
16774991811,4cyclictest4375-21diskmemload10:25:570
16785991711,3cyclictest4375-21diskmemload11:20:283
16785991710,5cyclictest1074-21idleruntime09:55:153
16785991710,5cyclictest1017-21expr12:20:143
16774991712,3cyclictest19399-21/usr/sbin/munin09:45:150
1678599166,8cyclictest0-21swapper/310:45:153
1678099168,6cyclictest16809-21/usr/sbin/munin07:10:151
1678099168,6cyclictest16809-21/usr/sbin/munin07:10:151
1678099157,6cyclictest2721-21expr09:05:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional