You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 02:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Thu Jun 11, 2026 12:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
78162182104,28sleep10-21swapper/107:09:231
75772170102,61sleep00-21swapper/007:06:170
76222163109,28sleep30-21swapper/307:06:533
76832161109,28sleep20-21swapper/207:07:392
190972710,4sleep10-21swapper/111:04:501
297312580,1sleep20-21swapper/210:24:012
297142580,3sleep00-21swapper/012:00:230
234272550,3sleep00-21swapper/012:20:170
245702530,2sleep10-21swapper/111:08:341
108952500,2sleep30-21swapper/308:30:143
221482490,1sleep20-21swapper/209:54:082
290442480,1sleep10-21swapper/110:23:201
209812470,1sleep10-21swapper/109:29:061
131822460,1sleep20-21swapper/211:48:292
174392450,3chrt0-21swapper/009:25:320
173672450,2sleep30-21swapper/309:25:273
72442420,1sleep30-21swapper/312:32:243
174162400,2sleep10-21swapper/107:30:161
52832370,2chrt0-21swapper/110:54:221
219562360,2chrt0-21swapper/210:17:552
215432350,2chrt0-21swapper/312:19:153
813899342,23cyclictest25385-21ssh10:45:111
63172320,2sleep00-21swapper/011:19:290
32092310,2sleep30-21swapper/311:40:233
813799298,5cyclictest12-21ksoftirqd/011:54:580
8137992919,7cyclictest28113-21/usr/sbin/munin07:55:140
813799281,25cyclictest0-21swapper/007:16:380
116132280,3sleep111610-21sh10:35:001
813799278,10cyclictest12-21ksoftirqd/010:50:560
814099267,11cyclictest34-21ksoftirqd/312:10:583
813999261,23cyclictest0-21swapper/207:16:292
8138992624,1cyclictest0-21swapper/107:16:321
8138992614,4cyclictest7645-21sh11:20:141
813799264,6cyclictest12-21ksoftirqd/010:35:160
8137992619,5cyclictest12-21ksoftirqd/010:50:000
149102260,1sleep30-21swapper/311:00:343
107992260,3sleep00-21swapper/010:34:110
814099254,9cyclictest34-21ksoftirqd/312:05:493
814099251,23cyclictest0-21swapper/307:17:263
814099251,13cyclictest34-21ksoftirqd/309:15:543
814099251,13cyclictest34-21ksoftirqd/309:15:533
813799259,3cyclictest141rcu_preempt11:29:180
813799259,2cyclictest141rcu_preempt10:57:460
813799258,4cyclictest12-21ksoftirqd/010:27:260
813799257,5cyclictest12-21ksoftirqd/010:16:090
813799254,12cyclictest12-21ksoftirqd/009:23:250
8137992513,4cyclictest12-21ksoftirqd/011:22:200
8137992512,4cyclictest12-21ksoftirqd/009:30:120
8137992512,2cyclictest37501rm09:40:120
5372250,1sleep20-21swapper/210:25:342
172972250,1sleep20-21swapper/211:50:532
814099245,11cyclictest34-21ksoftirqd/309:36:053
814099243,13cyclictest141rcu_preempt11:26:583
813999242,16cyclictest29418-21ssh12:25:122
8139992419,3cyclictest10662-21sh10:10:002
813799249,4cyclictest12-21ksoftirqd/012:17:560
813799248,4cyclictest12-21ksoftirqd/012:31:190
813799248,4cyclictest12-21ksoftirqd/011:01:050
813799247,9cyclictest12-21ksoftirqd/011:48:580
813799246,5cyclictest12-21ksoftirqd/010:22:290
8137992417,4cyclictest12-21ksoftirqd/009:20:000
8137992417,4cyclictest12-21ksoftirqd/009:20:000
8137992412,3cyclictest22094-21df11:55:120
8137992410,4cyclictest12-21ksoftirqd/009:56:080
300272240,11chrt22-21ksoftirqd/110:00:041
22142240,9sleep012-21ksoftirqd/012:28:590
814099237,8cyclictest141rcu_preempt09:53:463
814099232,11cyclictest34-21ksoftirqd/311:20:123
814099231,14cyclictest34-21ksoftirqd/310:02:213
8139992314,7cyclictest13439-21diskstats11:00:162
813899231,10cyclictest23800-21df_inode09:55:131
813799239,5cyclictest12-21ksoftirqd/010:05:090
813799237,4cyclictest12-21ksoftirqd/010:43:120
813799236,8cyclictest141rcu_preempt09:52:150
813799235,4cyclictest12-21ksoftirqd/011:12:070
8137992315,6cyclictest12-21ksoftirqd/009:10:000
8137992315,5cyclictest12-21ksoftirqd/009:40:010
8137992310,4cyclictest12-21ksoftirqd/012:10:250
8137992310,4cyclictest12-21ksoftirqd/011:41:280
306892230,2sleep20-21swapper/211:36:532
277462230,1sleep30-21swapper/311:35:133
241512230,1sleep20-21swapper/209:55:162
814099229,4cyclictest34-21ksoftirqd/309:32:563
814099227,4cyclictest34-21ksoftirqd/312:22:563
814099224,10cyclictest0-21swapper/310:33:143
814099222,10cyclictest34-21ksoftirqd/311:52:453
814099221,11cyclictest34-21ksoftirqd/311:33:523
814099220,20cyclictest24309-21ssh11:56:143
8139992216,4cyclictest8952-21sh12:10:002
8139992213,7cyclictest376-21memory08:05:172
8138992213,6cyclictest5166-21timerwakeupswit12:30:231
813799229,11cyclictest10713-21smartctl09:45:150
813799228,2cyclictest141rcu_preempt12:05:490
813799227,3cyclictest141rcu_preempt11:30:110
813799225,4cyclictest12-21ksoftirqd/009:14:500
8137992216,4cyclictest12-21ksoftirqd/007:45:000
8137992215,5cyclictest12-21ksoftirqd/012:40:010
8137992215,5cyclictest12-21ksoftirqd/008:35:000
8137992215,5cyclictest12-21ksoftirqd/008:25:000
8137992210,4cyclictest12-21ksoftirqd/010:03:410
234452220,2sleep2271rcuc/211:31:312
145882220,2sleep10-21swapper/112:38:041
108892220,2sleep20-21swapper/210:34:172
814099214,7cyclictest141rcu_preempt11:14:243
814099214,4cyclictest34-21ksoftirqd/310:58:373
814099211,5cyclictest524-21sh11:15:073
8140992113,6cyclictest26077-21fw_packets09:05:153
814099211,2cyclictest141rcu_preempt10:45:513
8140992111,8cyclictest12630-21ssh11:47:543
814099210,12cyclictest34-21ksoftirqd/309:49:373
813999212,13cyclictest4183-21kworker/u8:109:35:132
813999212,13cyclictest27501-21diskmemload10:40:142
813999211,9cyclictest170542ssh10:38:452
8139992114,5cyclictest3680-21cron08:15:002
8139992113,6cyclictest8358-21open_files11:20:212
8139992113,6cyclictest4699-21memory08:15:172
8139992113,6cyclictest26371-21memory09:05:192
813899218,9cyclictest17763-21ssh10:15:121
813899211,8cyclictest0-21swapper/111:40:441
8138992115,4cyclictest4948-21ls10:30:001
8138992114,5cyclictest27959-21tune2fs11:35:141
8138992113,6cyclictest14060-21seq10:35:431
8138992112,7cyclictest28346-21seq09:34:471
8138992112,6cyclictest560-21dbus-daemon08:10:001
813799215,8cyclictest141rcu_preempt11:05:390
813799212,13cyclictest32081-21apt-get08:05:110
8137992111,7cyclictest24272-21ls09:00:190
8137992111,7cyclictest20838-21seq08:52:180
8137992110,4cyclictest12-21ksoftirqd/011:38:370
814099206,4cyclictest34-21ksoftirqd/309:43:043
814099206,2cyclictest141rcu_preempt09:20:163
814099205,4cyclictest34-21ksoftirqd/310:19:083
814099201,6cyclictest11093-21ls12:35:183
814099201,2cyclictest34-21ksoftirqd/312:26:383
8140992011,6cyclictest278-21systemd-journal07:30:003
8140992010,8cyclictest53050irq/142-enp3s0-10:50:553
8140992010,7cyclictest5845-21systemctl08:20:003
814099200,9cyclictest2160-21fschecks_count08:10:143
814099200,5cyclictest23180-21ssh10:43:103
813999201,9cyclictest8047-21cron11:44:592
8139992013,5cyclictest29896-21sed08:00:002
8139992013,4cyclictest10219-21/usr/sbin/munin12:35:192
8139992012,6cyclictest278-21systemd-journal07:35:002
8139992012,6cyclictest23614-21seq11:07:372
8139992012,6cyclictest17351-21sensors_temp12:15:222
813899207,10cyclictest0-21swapper/111:45:121
8138992014,4cyclictest8889-21sh10:08:121
8138992013,5cyclictest9697-21vmstat07:10:251
8138992013,5cyclictest16021-21apt-get12:15:101
8138992013,4cyclictest14473-21grep11:25:141
8138992012,6cyclictest6569-21fw_packets08:20:151
8138992012,6cyclictest278-21systemd-journal12:00:001
8138992012,6cyclictest2560-21memory08:10:161
8138992012,6cyclictest20131-21pluginstate08:50:221
8138992011,7cyclictest8786-21df08:25:131
8138992011,7cyclictest8786-21df08:25:121
8138992011,7cyclictest15735-21cat12:15:001
8138992011,6cyclictest30862-21ntp_states08:00:201
813899201,13cyclictest2722-21sh12:05:121
8138992010,7cyclictest3077-21memory09:15:181
8138992010,7cyclictest3077-21memory09:15:181
8138992010,7cyclictest15365-21fw_packets08:40:161
813799207,3cyclictest141rcu_preempt10:11:010
813799203,12cyclictest1803-21apt-get08:10:120
8137992015,4cyclictest12-21ksoftirqd/007:50:000
8137992014,4cyclictest636-21runrttasks08:25:110
8137992014,4cyclictest636-21runrttasks08:25:110
8137992014,4cyclictest16212-21timerwakeupswit08:40:250
8137992014,4cyclictest12-21ksoftirqd/009:00:000
8137992014,4cyclictest12-21ksoftirqd/008:20:000
8137992012,6cyclictest17002-21seq08:45:030
8137992011,7cyclictest20968-21sed07:40:000
8137992011,6cyclictest278-21systemd-journal08:05:010
263632200,2sleep30-21swapper/309:56:373
814099199,8cyclictest28791-21rm10:23:053
814099199,8cyclictest19657-21rm11:05:103
814099199,7cyclictest23843-21df_inode07:45:143
814099199,7cyclictest17086-21df07:30:133
814099198,2cyclictest34-21ksoftirqd/308:45:123
814099195,4cyclictest34-21ksoftirqd/310:11:193
814099192,13cyclictest0-21swapper/309:10:113
8140991913,4cyclictest18878-21/usr/sbin/munin07:35:233
8140991913,3cyclictest6467-21sh10:05:453
8140991912,5cyclictest9072-21ntpq07:10:203
8140991911,6cyclictest32256-21awk10:25:213
8140991911,6cyclictest26291-21interrupts07:50:193
8140991911,6cyclictest16003-21readlink08:40:233
8140991911,6cyclictest13443-21cat08:35:183
814099191,13cyclictest19176-21apt-get08:50:123
8140991910,7cyclictest14272-21rm10:35:563
8139991913,4cyclictest4006-21cron10:05:002
8139991913,4cyclictest32559-21sh09:13:202
8139991912,4cyclictest12226-21pmu-power10:10:222
8139991911,6cyclictest20266-21sed07:35:232
8139991911,6cyclictest15214-21seq09:48:522
8139991911,6cyclictest14477-21grep07:25:002
8139991911,6cyclictest13660-21ntp_states08:35:212
8139991911,6cyclictest12612-21cat08:35:012
8139991910,7cyclictest9380-21ntp_states08:25:202
8139991910,7cyclictest9380-21ntp_states08:25:192
8139991910,7cyclictest560-21dbus-daemon07:55:002
8139991910,7cyclictest32638-21cron11:15:002
8139991910,7cyclictest22787-21tune2fs09:30:142
8139991910,6cyclictest3766-21grep12:30:122
8139991910,6cyclictest2864-21grep09:15:162
8139991910,6cyclictest2864-21grep09:15:152
8139991910,6cyclictest21990-21cut08:55:172
8139991910,6cyclictest14900-21grep11:25:182
813899192,12cyclictest25652-21apt-get07:50:131
813899191,7cyclictest15452-21df_inode11:50:131
8138991913,4cyclictest6241-21sh10:55:071
8138991913,4cyclictest32200-21sh09:36:421
8138991913,4cyclictest2859-21sh12:29:391
8138991913,4cyclictest26854-21sh11:34:581
8138991913,4cyclictest21348-21/usr/sbin/munin07:40:151
8138991913,3cyclictest22725-21unixbench_multi08:55:241
8138991911,6cyclictest28839-21ls09:10:181
8138991911,6cyclictest1735-21awk11:15:201
8138991911,5cyclictest52950irq/141-enp3s0-09:45:111
8138991910,6cyclictest52950irq/141-enp3s0-12:00:131
8138991910,6cyclictest52950irq/141-enp3s0-10:10:111
8138991910,6cyclictest27167-21latency11:10:171
8137991910,6cyclictest9393-21sed07:10:230
310952190,3sleep20-21swapper/208:00:232
8140991812,4cyclictest34-21ksoftirqd/307:25:003
8140991811,5cyclictest22624-21ls08:55:243
8140991810,6cyclictest32260-21cat08:05:123
8140991810,6cyclictest30183-21awk12:00:263
8140991810,6cyclictest24049-21cat09:00:173
8140991810,6cyclictest21934-21which07:40:163
813999189,6cyclictest278-21systemd-journal11:20:002
8139991813,3cyclictest7440-21open_inodes10:55:212
8139991813,3cyclictest3414-21sh09:40:092
8139991812,4cyclictest875-21sh12:03:352
8139991811,5cyclictest32365-21expr10:50:132
8139991811,5cyclictest24199-21sed12:20:252
8139991811,5cyclictest24135-21cat07:45:162
8139991811,5cyclictest21426-21awk07:40:122
8139991811,5cyclictest15429-21sed07:25:162
8139991810,6cyclictest8778-21cat07:10:172
8139991810,6cyclictest28042-21pluginstate12:00:002
8139991810,6cyclictest23450-21find09:00:102
8139991810,6cyclictest19021-21sed08:50:012
8139991810,6cyclictest15800-21ntp_kernel_pll_08:40:192
8139991810,6cyclictest10271-21grep09:20:232
813899182,13cyclictest27492-21sh09:10:001
813899182,11cyclictest12484-21apt-get07:20:101
813899182,11cyclictest10600-21apt-get08:30:131
8138991813,3cyclictest28714-21sh07:55:211
8138991813,3cyclictest25773-21sh12:21:461
8138991812,4cyclictest8886-21sh09:20:101
8138991812,4cyclictest22857-21sh10:42:511
8138991812,4cyclictest15689-21processes07:25:211
8138991811,5cyclictest17162-21awk09:50:151
8138991810,6cyclictest4007-21cat08:15:121
813799185,3cyclictest141rcu_preempt07:25:210
814099172,11cyclictest6106-21apt-get08:20:133
8140991710,5cyclictest8673-21ls08:25:123
8140991710,5cyclictest8673-21ls08:25:123
8140991710,5cyclictest28029-21expr07:55:133
813999179,6cyclictest20403-21cat08:50:232
813999172,11cyclictest27103-21users10:45:252
813999171,7cyclictest9772-21/usr/sbin/munin12:10:152
8139991712,3cyclictest19403-21sh09:27:292
8139991710,5cyclictest6417-21expr08:20:142
813899171,12cyclictest17036-21apt-get08:45:101
813899171,11cyclictest3464-21apt-get09:40:121
813899171,11cyclictest18903-21apt-get07:35:121
8138991710,5cyclictest25317-21seq07:49:101
8138991710,5cyclictest23624-21expr09:00:131
8138991710,5cyclictest12796-21awk08:35:121
813799178,6cyclictest26159-21gltestperf07:50:170
813799177,2cyclictest141rcu_preempt08:35:190
814099168,5cyclictest31223-21timerandwakeup08:00:253
813799164,2cyclictest141rcu_preempt07:20:100
8137991610,4cyclictest17076-21expr07:30:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional