You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 18:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackaslot8s.osadl.org (updated Tue Apr 14, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
302672222199,16sleep00-21swapper/007:05:560
302782179104,26sleep10-21swapper/107:06:061
1988521500,11sleep228-21ksoftirqd/211:37:162
30435214596,24sleep20-21swapper/207:08:072
305202139102,26sleep30-21swapper/307:09:123
23482710,2sleep10-21swapper/109:21:361
253962690,2chrt0-21swapper/210:52:382
170692650,2sleep10-21swapper/109:57:001
77412630,2sleep20-21swapper/211:52:412
228672630,3sleep10-21swapper/112:29:051
59302610,2sleep20-21swapper/210:37:582
58372600,2sleep30-21swapper/310:13:273
58372600,2sleep30-21swapper/310:13:273
109812600,2sleep10-21swapper/110:16:571
108772600,1sleep10-21swapper/109:52:251
252812590,2sleep10-21swapper/111:16:541
8972580,2sleep00-21swapper/012:36:030
94552560,2sleep00-21swapper/008:50:010
158922560,2sleep10-21swapper/111:34:591
85432550,2sleep3331rcuc/308:45:233
279252550,2sleep20-21swapper/212:08:032
19172550,2sleep20-21swapper/210:10:172
19172550,2sleep20-21swapper/210:10:162
235022500,1sleep10-21swapper/110:50:411
67772490,2chrt0-21swapper/110:14:251
67772490,2chrt0-21swapper/110:14:241
84322470,2sleep20-21swapper/212:17:462
277332470,1sleep10-21swapper/111:43:301
277332470,1sleep10-21swapper/111:43:291
98832460,4sleep39884-21uniq08:50:153
23782420,1sleep10-21swapper/111:24:511
77492400,1sleep10-21swapper/111:04:141
68102340,2chrt0-21swapper/010:38:520
300092330,2chrt0-21swapper/009:18:340
134672330,11sleep012-21ksoftirqd/012:21:100
265972310,2sleep00-21swapper/010:53:530
230912300,11sleep012-21ksoftirqd/011:15:190
134032270,2sleep30-21swapper/310:19:293
31046992612,9cyclictest30208-21ssh12:10:102
3104299261,2cyclictest0-21swapper/007:10:070
3104999250,23cyclictest0-21swapper/307:10:013
3104699252,15cyclictest8933-21ssh11:05:122
3104699251,23cyclictest0-21swapper/207:10:092
167682250,1sleep30-21swapper/309:32:423
31046992417,5cyclictest28-21ksoftirqd/210:10:002
31046992413,7cyclictest53050irq/142-enp3s0-11:30:032
302882240,1sleep00-21swapper/010:06:570
31049992314,6cyclictest7125-21cron08:45:003
31049992313,7cyclictest6685-21fw_packets09:25:163
31046992314,7cyclictest14369-21fw_packets09:55:172
31046992312,8cyclictest5697-21cron09:24:592
31045992314,7cyclictest17546-21ls09:05:251
3104299232,14cyclictest17613-21apt-get12:25:130
31042992310,3cyclictest4217-21mailstats11:25:230
233682230,1sleep30-21swapper/310:01:393
83962220,9sleep228-21ksoftirqd/211:29:182
56902220,1sleep00-21swapper/009:24:590
31045992217,3cyclictest549-21dump-pmu-power07:15:001
31045992213,6cyclictest24747-21cron09:15:001
31045992211,8cyclictest3813-21apt-get11:50:121
31042992212,7cyclictest21424-21users09:35:280
31042992211,7cyclictest21092-21tr10:25:140
302482220,1sleep00-21swapper/010:31:260
216812220,2sleep30-21swapper/311:14:593
3104999219,9cyclictest3390-21cron11:49:593
31049992113,6cyclictest14401-21ssh12:22:073
31049992112,7cyclictest28072-21cat08:15:253
31049992112,6cyclictest18010-21cat12:25:153
3104699213,11cyclictest18060-21diskmemload10:45:132
3104699212,14cyclictest16004-21taskset07:50:122
3104699212,13cyclictest4736-21taskset07:23:042
31046992114,5cyclictest6558-21ls08:40:242
31046992112,7cyclictest20161-21tune2fs09:35:172
3104599211,7cyclictest25919-21latency_hist09:40:001
31045992113,6cyclictest560-21dbus-daemon12:05:011
31045992113,6cyclictest17517-21seq07:50:261
31045992112,7cyclictest6560-21ls07:25:261
3104599211,18cyclictest28897-21ssh12:09:041
3104599210,6cyclictest2996-21wc11:25:141
3104299214,5cyclictest636-21runrttasks09:10:570
31042992116,4cyclictest12-21ksoftirqd/012:05:010
31042992113,6cyclictest9309-21wc07:35:000
31042992113,6cyclictest6639-21tune2fs09:25:170
31042992112,6cyclictest30455-21cat11:45:190
244012210,2sleep00-21swapper/012:30:130
182492210,2sleep20-21swapper/210:22:402
115212210,1sleep00-21swapper/011:55:200
85222200,9sleep334-21ksoftirqd/309:50:243
31049992013,5cyclictest9404-21ntpq10:40:223
31049992013,5cyclictest14913-21open_files09:55:213
31049992013,4cyclictest29428-21sh10:06:053
31049992012,6cyclictest6100-21ntp_states07:25:223
31049992012,6cyclictest4954-21perl12:15:163
31049992012,6cyclictest19457-21seq11:36:513
31049992012,6cyclictest15126-21cat10:45:133
31049992011,7cyclictest5530-21users11:50:263
31049992011,7cyclictest20911-21seq10:25:113
31049992010,6cyclictest52850irq/140-enp3s0-11:30:033
3104699205,10cyclictest28-21ksoftirqd/209:10:122
3104699202,13cyclictest22156-21apt-get11:15:122
31046992013,5cyclictest28-21ksoftirqd/208:29:092
31046992012,6cyclictest9913-21sed10:40:252
31046992012,6cyclictest7918-21grep10:15:132
31046992011,7cyclictest25465-21seq10:03:492
31046992010,6cyclictest53050irq/142-enp3s0-11:00:112
3104599202,13cyclictest11052-21ssh12:20:131
31045992014,4cyclictest7235-21sh12:16:341
31045992014,4cyclictest10256-21sh10:40:371
31045992012,6cyclictest4459-21seq08:36:121
31045992012,6cyclictest4459-21seq08:36:121
31045992012,6cyclictest30944-21seq10:56:341
31045992012,6cyclictest2490-21seq12:13:261
31045992012,6cyclictest22787-21forks08:05:141
31045992012,6cyclictest16118-21cat09:05:011
31045992012,6cyclictest13663-21seq08:58:591
31045992012,6cyclictest12002-21grep11:55:251
3104599201,17cyclictest30907-21rm09:43:191
31045992011,7cyclictest26844-21date10:05:001
3104299209,7cyclictest278-21systemd-journal11:35:050
3104299202,13cyclictest28874-21apt-get08:20:130
3104299201,6cyclictest24530-21ls11:40:270
3104299201,6cyclictest24530-21ls11:40:270
3104299201,4cyclictest0-21swapper/011:13:210
31042992012,5cyclictest28793-21date08:20:000
289172200,1sleep00-21swapper/009:41:180
20802200,1sleep30-21swapper/312:13:023
31049991914,3cyclictest2540-21sh12:37:433
31049991913,4cyclictest28861-21sh12:09:023
31049991913,4cyclictest26250-21sh12:30:323
31049991913,4cyclictest25685-21sh09:39:463
31049991912,5cyclictest31715-21ssh10:32:563
31049991912,5cyclictest31635-21cat08:25:143
31049991912,5cyclictest24685-21seq09:14:573
31049991911,6cyclictest9987-21expr07:35:183
31049991911,6cyclictest3332-21cut08:35:173
31049991911,6cyclictest3332-21cut08:35:163
31049991911,6cyclictest29594-21cat11:20:203
31049991911,6cyclictest18270-21expr07:55:133
31049991911,6cyclictest16106-21seq11:59:303
3104999191,15cyclictest15923-21munin-run09:05:013
3104999191,13cyclictest601-21ssh09:45:133
31049991910,7cyclictest26643-21cat08:15:013
31049991910,5cyclictest52850irq/140-enp3s0-11:40:153
31049991910,5cyclictest52850irq/140-enp3s0-11:40:143
3104699198,7cyclictest53050irq/142-enp3s0-11:55:122
3104699194,8cyclictest569-21apt-get09:45:132
31046991913,4cyclictest28-21ksoftirqd/212:29:592
31046991913,4cyclictest13763-21basename09:00:012
31046991912,5cyclictest0-21swapper/211:40:242
31046991912,5cyclictest0-21swapper/211:40:232
31046991912,4cyclictest18060-21diskmemload12:39:442
31046991911,6cyclictest18059-21ls09:10:002
31046991911,6cyclictest12640-21ntp_states07:40:212
31046991910,6cyclictest278-21systemd-journal12:05:012
31046991910,6cyclictest0-21swapper/210:25:142
3104599192,12cyclictest20299-21apt-get08:00:131
3104599192,12cyclictest1739-21apt-get10:35:121
31045991914,3cyclictest18334-21sh11:35:431
31045991912,5cyclictest4724-21seq12:40:001
31045991912,5cyclictest27893-21ls08:15:241
31045991912,5cyclictest18618-21grep07:55:151
31045991912,5cyclictest11227-21seq07:37:181
31045991911,6cyclictest29848-21expr11:45:131
31045991911,6cyclictest27983-21entropy10:30:171
31045991911,6cyclictest2741-21cut08:35:011
31045991911,6cyclictest27139-21awk10:05:131
31045991911,6cyclictest15338-21seq11:10:031
31045991911,6cyclictest14803-21open_inodes07:45:211
31045991911,5cyclictest21259-21cat10:25:141
31045991910,7cyclictest11834-21grep07:40:131
3104299199,7cyclictest8118-21df10:15:140
3104299199,7cyclictest14228-21grep09:30:230
3104299192,3cyclictest141rcu_preempt10:10:120
3104299192,3cyclictest141rcu_preempt10:10:110
3104299191,9cyclictest0-21swapper/010:23:540
3104299191,5cyclictest0-21swapper/009:59:410
3104299191,4cyclictest0-21swapper/010:45:410
31042991912,5cyclictest6358-21tr08:40:210
31042991912,5cyclictest4790-21df12:15:150
31042991912,5cyclictest2441-21seq08:32:480
31042991912,4cyclictest20017-21sh08:00:010
31042991911,6cyclictest5912-21sed07:25:180
31042991911,6cyclictest5054-21date07:25:010
31042991911,6cyclictest30589-21df12:10:130
31042991911,6cyclictest20611-21cut10:00:150
31042991911,6cyclictest13806-21sed07:45:010
31042991911,6cyclictest10191-21tr11:05:210
31042991911,5cyclictest7931-21awk09:50:180
3104299190,7cyclictest636-21runrttasks09:48:490
134282190,9sleep228-21ksoftirqd/212:21:082
100542190,2chrt0-21swapper/209:27:392
3104999189,7cyclictest25397-21cstates09:15:143
3104999189,7cyclictest1565-21ls08:30:213
31049991813,3cyclictest4915-21sh10:36:553
31049991813,3cyclictest28761-21sh10:55:163
31049991813,3cyclictest27818-21sh09:40:273
31049991813,3cyclictest12554-21needreboot07:40:193
31049991812,4cyclictest24796-21sh10:52:013
31049991812,4cyclictest11425-21sh11:06:123
31049991812,3cyclictest16864-21sh12:00:023
31049991811,5cyclictest8150-21needreboot07:30:193
31049991811,5cyclictest3072-21grep11:25:153
31049991811,5cyclictest21075-21cat08:00:183
31049991811,5cyclictest15181-21date10:20:173
31049991811,5cyclictest11743-21readlink08:55:013
31049991811,4cyclictest18060-21diskmemload09:23:223
3104999181,12cyclictest18060-21diskmemload11:15:123
31049991810,6cyclictest2556-21expr11:00:143
31049991810,6cyclictest22512-21ls08:05:133
3104699189,6cyclictest10130-21sed07:35:182
3104699182,12cyclictest29616-21apt-get11:45:132
3104699182,11cyclictest0-21swapper/211:20:142
3104699181,7cyclictest19055-21munin-run09:35:002
3104699181,5cyclictest0-21swapper/211:14:072
31046991814,2cyclictest29446-21sensors_temp10:55:242
31046991813,3cyclictest3216-21sh08:35:162
31046991813,3cyclictest3216-21sh08:35:152
31046991813,3cyclictest26934-21gltestperf09:40:172
31046991813,3cyclictest13397-21cron09:55:002
31046991811,5cyclictest29301-21cut08:20:162
31046991810,6cyclictest278-21systemd-journal07:35:002
31046991810,6cyclictest15044-21ls07:45:232
31046991810,6cyclictest10487-21cut08:50:222
3104699180,5cyclictest30010-21ssh09:18:342
3104599189,7cyclictest24715-21expr12:30:151
31045991812,4cyclictest28441-21rm09:16:581
31045991812,4cyclictest16814-21sh10:21:111
3104599181,15cyclictest6761-21sed09:25:161
31045991811,5cyclictest3225-21expr07:20:131
31045991811,5cyclictest15280-21expr10:45:141
3104599181,14cyclictest8732-21ssh11:05:011
31045991810,6cyclictest9480-21cat08:50:001
31045991810,6cyclictest5448-21seq09:48:401
31045991810,6cyclictest25718-21grep08:10:241
31045991810,6cyclictest13583-21expr09:30:171
3104299189,6cyclictest9187-21grep10:40:200
3104299182,9cyclictest3842-21ssh11:50:130
3104299182,11cyclictest9527-21apt-get08:50:130
3104299182,11cyclictest22437-21apt-get08:05:130
3104299181,6cyclictest0-21swapper/010:55:120
31042991811,5cyclictest21002-21sed08:00:170
31042991811,5cyclictest16817-21sed07:50:180
31042991810,6cyclictest29159-21df11:20:170
31042991810,5cyclictest25250-21cut08:10:180
3104999179,6cyclictest4076-21gzip07:20:223
31049991712,3cyclictest799-21/usr/sbin/munin07:15:093
31049991711,4cyclictest18950irq/143-ahci[0007:50:143
31049991710,5cyclictest29149-21expr08:20:153
31049991710,5cyclictest17598-21grep09:05:263
3104699179,5cyclictest8056-21sed08:45:182
3104699179,5cyclictest53050irq/142-enp3s0-12:30:142
3104699175,9cyclictest560-21dbus-daemon08:00:012
3104699174,3cyclictest141rcu_preempt07:20:002
3104699171,5cyclictest0-21swapper/210:31:342
3104699171,5cyclictest0-21swapper/208:07:182
31046991712,3cyclictest26447-21cron08:15:002
31046991710,5cyclictest5882-21cat07:25:182
31046991710,5cyclictest15132-21run-parts09:00:212
3104599179,6cyclictest6641-21grep08:40:251
3104599179,5cyclictest8504-21switchtime07:30:231
31045991712,3cyclictest636-21runrttasks07:15:201
31045991711,4cyclictest0-21swapper/108:25:131
31045991710,5cyclictest10230-21cat08:50:191
3104299179,5cyclictest0-21swapper/007:35:240
3104299172,11cyclictest31011-21apt-get08:25:110
3104299172,11cyclictest25167-21ssh12:05:250
3104299171,5cyclictest0-21swapper/011:31:510
31042991712,3cyclictest15916-21latency_hist09:05:000
31042991711,4cyclictest13770-21dump-pmu-power09:00:010
31042991710,4cyclictest898-21cpuspeed_turbos07:15:100
27402170,4sleep02745-21fschecks_count11:00:150
3104999169,5cyclictest15212-21cat07:45:253
31046991610,4cyclictest26787-21cpuspeed_turbos08:15:152
31046991610,3cyclictest21012-21latency08:00:172
3104599168,5cyclictest29340-21gltestperf08:20:171
31042991611,3cyclictest2857-21/usr/sbin/munin08:35:180
31042991611,3cyclictest2857-21/usr/sbin/munin08:35:170
31042991611,3cyclictest17101-21open_files09:05:220
31042991611,3cyclictest14432-21hwlatdetect07:45:170
3104699158,4cyclictest1731-21pmu-power08:30:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional