You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-21 - 08:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sun Jun 21, 2026 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12878217598,25sleep10-21swapper/119:09:541
126622175103,28sleep20-21swapper/219:07:062
128622166103,27sleep00-21swapper/019:09:410
125532160103,29sleep30-21swapper/319:05:433
76612630,2sleep10-21swapper/122:03:321
228002570,1sleep00-21swapper/000:14:300
58982560,2sleep3331rcuc/320:05:253
326832530,1sleep00-21swapper/000:20:520
95482500,2sleep00-21swapper/022:05:120
101272500,2sleep00-21swapper/000:05:070
90082490,1sleep10-21swapper/121:40:171
24462480,2sleep20-21swapper/200:23:032
13161994810,20cyclictest32263-21diskmemload21:40:142
48812460,1sleep30-21swapper/321:37:023
155642460,1sleep30-21swapper/300:32:393
290322440,2chrt0-21swapper/323:06:273
1316199449,22cyclictest636-21runrttasks23:52:072
13161994410,20cyclictest32263-21diskmemload23:43:522
166462430,2sleep10-21swapper/123:21:141
98642420,1sleep30-21swapper/322:05:153
221552420,1sleep20-21swapper/200:13:522
1316199429,21cyclictest32263-21diskmemload23:37:242
1316199421,7cyclictest32263-21diskmemload22:51:472
28682400,2sleep1211rcuc/121:35:231
13161993910,16cyclictest32263-21diskmemload22:47:512
13161993910,16cyclictest32263-21diskmemload22:47:512
1316199388,17cyclictest32263-21diskmemload23:04:432
1316199383,31cyclictest0-21swapper/223:20:122
1316199363,3cyclictest0-21swapper/223:15:132
13161993612,21cyclictest28687-21perl21:00:152
1316199360,6cyclictest4944-21rm23:13:012
1316199360,6cyclictest32263-21diskmemload23:56:432
1316199359,15cyclictest32263-21diskmemload21:51:572
1316199352,26cyclictest9589-21apt-get22:05:142
13161993511,21cyclictest13028-21cut22:55:162
13161993510,22cyclictest24320-21cut20:50:152
1316199350,4cyclictest21545-21sh00:36:542
95312340,11sleep012-21ksoftirqd/020:15:170
1316199341,29cyclictest0-21swapper/222:00:132
13161993411,20cyclictest28463-21expr19:45:122
1316199341,11cyclictest26164-21ssh23:05:062
1316199340,7cyclictest3977-21ssh21:12:242
1316199340,6cyclictest5125-21latency22:25:172
13161993310,20cyclictest26908-21grep20:55:192
1316199330,5cyclictest23242-21grep19:30:262
1316199330,4cyclictest25433-21grep00:15:252
1316199330,3cyclictest17205-21ssh21:45:492
1316199329,20cyclictest16765-21unixbench_multi20:30:232
1316199328,21cyclictest9906-21awk20:15:222
1316199327,22cyclictest21194-21cat21:25:172
1316199326,23cyclictest3942-21fschecks_time00:00:152
1316199325,23cyclictest18206-21fschecks_count22:35:142
1316199324,23cyclictest24744-21apt-get22:40:122
1316199322,25cyclictest0-21swapper/219:35:122
1316199322,25cyclictest0-21swapper/219:35:122
1316199321,7cyclictest18086-21ssh21:22:592
1316199321,28cyclictest11169-21grep20:20:132
1316199321,1cyclictest0-21swapper/220:08:022
1316199321,1cyclictest0-21swapper/219:28:162
1316199321,10cyclictest27590-21ssh23:30:112
1316199321,10cyclictest21951-21/usr/sbin/munin23:25:222
1316299310,7cyclictest34-21ksoftirqd/323:00:003
1316299310,6cyclictest6587-21munin-run21:15:003
1316199317,21cyclictest16571-21latency_hist00:10:002
1316199317,20cyclictest5434-21sed00:25:162
1316199312,6cyclictest31366-21ssh22:20:242
1316199311,6cyclictest28104-21ssh21:30:182
1316199311,27cyclictest1582-21ssh21:35:122
1316199311,10cyclictest7845-21sort21:15:182
1316199311,10cyclictest7845-21sort21:15:182
1316199310,3cyclictest13476-21sh00:30:332
1316199307,20cyclictest4364-21date20:05:012
1316199302,8cyclictest15388-21/usr/sbin/munin23:45:212
13161993022,6cyclictest0-21swapper/219:43:292
1316199301,5cyclictest18263-21/usr/sbin/munin19:20:182
1316199301,4cyclictest75-21kswapd022:30:122
13161993010,17cyclictest32263-21diskmemload21:58:452
1316299297,11cyclictest34-21ksoftirqd/323:59:313
1316299291,14cyclictest34-21ksoftirqd/322:43:473
1316199297,19cyclictest278-21systemd-journal19:14:552
1316199294,22cyclictest20374-21wc20:40:172
1316199291,9cyclictest23198-21/usr/sbin/munin22:15:242
1316199291,9cyclictest13677-21tune2fs20:25:162
1316199291,7cyclictest21554-21ssh22:13:572
1315999290,4cyclictest636-21runrttasks19:59:580
45662280,4chrt0-21swapper/022:00:260
232002280,2chrt0-21swapper/323:02:343
1316299287,11cyclictest34-21ksoftirqd/322:50:253
1316299286,11cyclictest34-21ksoftirqd/323:14:103
1316299280,8cyclictest34-21ksoftirqd/322:40:003
1316199286,19cyclictest230062sleep220:45:252
1316199281,8cyclictest32266-21dump-pmu-power21:10:002
1316199280,4cyclictest7150-21fschecks_count20:10:152
1316099281,25cyclictest0-21swapper/120:32:041
1315999281,25cyclictest0-21swapper/020:31:080
3952270,2sleep10-21swapper/119:55:151
1316299278,10cyclictest34-21ksoftirqd/321:28:023
1316299277,10cyclictest34-21ksoftirqd/321:47:543
1316299276,9cyclictest34-21ksoftirqd/323:31:103
1316299271,24cyclictest0-21swapper/320:04:193
1316299271,24cyclictest0-21swapper/319:34:123
1316199271,8cyclictest18075-21/usr/sbin/munin20:35:162
1316199271,8cyclictest16239-21awk19:15:202
1316199271,7cyclictest32635-21cpuspeed_turbos19:55:132
13160992724,2cyclictest0-21swapper/120:02:021
1316099271,25cyclictest0-21swapper/119:37:321
1316099271,25cyclictest0-21swapper/119:37:321
1316099271,24cyclictest0-21swapper/120:16:221
1316099271,24cyclictest0-21swapper/120:05:201
1316099271,24cyclictest0-21swapper/119:51:451
1316099271,24cyclictest0-21swapper/119:31:481
1316099271,24cyclictest0-21swapper/119:20:321
1315999279,12cyclictest141rcu_preempt21:24:230
1315999271,24cyclictest0-21swapper/020:36:040
1315999271,24cyclictest0-21swapper/020:29:130
1315999271,24cyclictest0-21swapper/020:21:120
1315999271,24cyclictest0-21swapper/020:00:330
1315999271,24cyclictest0-21swapper/019:42:520
1315999271,24cyclictest0-21swapper/019:27:360
1315999271,24cyclictest0-21swapper/019:21:190
1316299269,4cyclictest34-21ksoftirqd/323:51:513
1316299268,9cyclictest34-21ksoftirqd/323:21:043
1316299266,10cyclictest34-21ksoftirqd/321:51:383
1316299264,10cyclictest34-21ksoftirqd/300:25:493
1316299263,5cyclictest24728-21ls22:15:243
1316299262,6cyclictest34-21ksoftirqd/321:15:253
1316299262,6cyclictest34-21ksoftirqd/321:15:253
1316299261,6cyclictest0-21swapper/322:20:133
1316299261,23cyclictest0-21swapper/320:26:493
1316299261,23cyclictest0-21swapper/320:10:463
1316299261,23cyclictest0-21swapper/319:57:483
1316299261,23cyclictest0-21swapper/319:51:213
1316299261,23cyclictest0-21swapper/319:45:363
1316299261,23cyclictest0-21swapper/319:36:103
1316299261,23cyclictest0-21swapper/319:36:103
1316299261,1cyclictest0-21swapper/320:31:033
1316299261,1cyclictest0-21swapper/319:21:143
1316299260,7cyclictest34-21ksoftirqd/322:25:123
1316199261,7cyclictest30527-21cstates19:50:132
13160992623,2cyclictest0-21swapper/120:28:471
1316099261,2cyclictest0-21swapper/120:11:261
1316099261,2cyclictest0-21swapper/119:49:211
1316099261,1cyclictest0-21swapper/119:41:411
1316099261,1cyclictest0-21swapper/119:25:211
13159992624,1cyclictest0-21swapper/019:52:310
13159992624,1cyclictest0-21swapper/019:45:160
13159992623,2cyclictest0-21swapper/020:06:020
1315999261,24cyclictest0-21swapper/019:35:050
1315999261,24cyclictest0-21swapper/019:35:050
1315999260,24cyclictest0-21swapper/019:32:130
93472250,2sleep10-21swapper/121:16:011
93472250,2sleep10-21swapper/121:16:011
31902250,0sleep0131rcuc/022:24:180
1316299257,4cyclictest34-21ksoftirqd/321:34:263
1316299257,3cyclictest141rcu_preempt00:13:393
1316299256,11cyclictest34-21ksoftirqd/321:23:103
1316299253,9cyclictest34-21ksoftirqd/300:23:223
13162992523,1cyclictest0-21swapper/319:42:373
13162992522,2cyclictest0-21swapper/320:20:073
13162992520,3cyclictest331rcuc/323:41:453
1316299251,9cyclictest34-21ksoftirqd/300:40:003
1316299251,5cyclictest34-21ksoftirqd/323:15:143
1316299250,6cyclictest278-21systemd-journal19:20:003
1316299250,23cyclictest0-21swapper/320:16:393
1316299250,23cyclictest0-21swapper/319:25:163
13160992522,2cyclictest0-21swapper/120:24:141
13160992512,9cyclictest21231-21df23:25:151
1315999253,17cyclictest21821-21ssh21:50:130
13159992523,1cyclictest0-21swapper/020:11:240
1315999252,10cyclictest304550mandb00:00:010
1316299248,4cyclictest34-21ksoftirqd/323:28:423
1316299247,9cyclictest34-21ksoftirqd/300:08:363
1316299244,9cyclictest34-21ksoftirqd/300:02:413
1316299241,14cyclictest34-21ksoftirqd/322:11:593
83182230,1sleep10-21swapper/100:03:321
1316299238,3cyclictest34-21ksoftirqd/323:49:533
1316299236,9cyclictest34-21ksoftirqd/322:30:183
1316299232,8cyclictest34-21ksoftirqd/323:35:183
13162992320,2cyclictest331rcuc/300:15:273
1316299231,6cyclictest34-21ksoftirqd/322:00:023
13162992310,4cyclictest34-21ksoftirqd/321:40:143
1316299230,6cyclictest34-21ksoftirqd/322:45:123
1316299230,6cyclictest34-21ksoftirqd/322:45:123
1316099232,14cyclictest26166-21ssh23:05:061
1316099231,11cyclictest289182wc21:00:171
13159992314,7cyclictest21047-21users23:00:250
1315999230,11cyclictest206002sh23:48:520
302172220,6chrt0-21swapper/121:55:261
1316299225,9cyclictest34-21ksoftirqd/321:56:513
13162992217,3cyclictest34-21ksoftirqd/321:00:013
13160992220,1cyclictest0-21swapper/120:35:491
13160992211,7cyclictest23078-21sh22:15:031
1315999227,5cyclictest12-21ksoftirqd/022:45:180
1315999227,5cyclictest12-21ksoftirqd/022:45:180
13159992215,5cyclictest560-21dbus-daemon23:55:000
13159992214,5cyclictest24439-21sed22:40:010
1315999221,18cyclictest19655-21apt-get20:40:110
1316299215,10cyclictest34-21ksoftirqd/320:44:263
13162992115,4cyclictest34-21ksoftirqd/320:50:003
1316299210,12cyclictest0-21swapper/321:02:513
13160992113,6cyclictest7035-21cut23:15:011
13160992112,7cyclictest24570-21ssh22:40:021
13160992112,7cyclictest14183-21diskstats21:20:151
1315999211,5cyclictest32263-21diskmemload21:12:010
13159992115,3cyclictest12-21ksoftirqd/020:55:160
13159992113,6cyclictest6158-21sensors_temp00:25:230
13159992112,7cyclictest9915-21ls21:40:250
13159992110,9cyclictest30034-21ssh00:19:590
1316299207,4cyclictest34-21ksoftirqd/319:14:583
13162992018,1cyclictest0-21swapper/320:36:323
13162992016,2cyclictest331rcuc/320:53:553
1316299200,8cyclictest34-21ksoftirqd/321:05:173
13160992015,3cyclictest3923-21idleruntime-cro22:25:001
13160992014,4cyclictest52950irq/141-enp3s0-23:00:221
13160992013,5cyclictest30675-21ls23:55:241
13160992012,6cyclictest749-21tune2fs21:10:161
13160992012,6cyclictest560-21dbus-daemon21:50:001
13160992012,6cyclictest26776-21sed20:55:171
13160992012,6cyclictest22344-21seq21:25:311
13160992012,6cyclictest17338-21cron22:35:001
13160992012,6cyclictest17108-21ls22:10:191
13160992012,6cyclictest14841-21seq00:31:561
13160992012,6cyclictest13439-21date22:55:201
13160992011,7cyclictest8745-21awk23:40:131
13160992011,6cyclictest9080-21awk23:15:271
13160992011,6cyclictest52950irq/141-enp3s0-22:05:101
1316099201,14cyclictest4863-21rm00:25:131
1316099200,4cyclictest0-21swapper/122:52:511
1315999207,10cyclictest573-21ssh23:10:090
1315999204,3cyclictest141rcu_preempt23:43:230
1315999202,13cyclictest23082-21ssh22:15:030
13159992014,4cyclictest17005-21sh22:58:070
13159992012,6cyclictest8465-21timerandwakeup21:15:250
13159992012,6cyclictest8465-21timerandwakeup21:15:250
13159992011,7cyclictest27708-21cut23:30:120
13159992011,7cyclictest1949-21df21:35:140
13159992010,5cyclictest0-21swapper/023:55:110
1316099191,9cyclictest31160-21hddtemp_smartct00:20:191
13160991914,3cyclictest17698-21sh22:35:081
13160991913,4cyclictest22886-21sendmail_mailst20:45:231
13160991913,4cyclictest17233-21idleruntime-cro19:20:001
13160991912,5cyclictest680-21seq21:34:341
13160991912,4cyclictest32263-21diskmemload23:38:321
13160991912,4cyclictest27023-21sh00:17:001
13160991911,5cyclictest32263-21diskmemload21:52:461
1316099191,14cyclictest22213-21ssh23:50:051
1315999199,6cyclictest53050irq/142-enp3s0-21:45:120
1315999199,2cyclictest12-21ksoftirqd/000:30:290
1315999196,5cyclictest12-21ksoftirqd/021:25:250
1315999192,12cyclictest15344-21apt-get19:15:110
13159991914,3cyclictest4313-21sh23:36:050
13159991913,4cyclictest28370-21pmu-power21:30:210
13159991913,4cyclictest25835-21open_files22:40:210
13159991913,4cyclictest16560-21sh23:21:080
13159991913,3cyclictest28400-21sh21:55:060
13159991912,5cyclictest7961-21tail23:15:170
13159991912,5cyclictest30112-21sh21:05:000
13159991911,6cyclictest7330-21wc22:50:260
13159991911,6cyclictest5699-21cat22:25:230
13159991911,4cyclictest12-21ksoftirqd/023:30:010
13159991910,7cyclictest31524-21swap21:05:220
1316099189,7cyclictest28359-21expr23:30:161
1316099188,7cyclictest32759-21grep22:45:231
1316099188,7cyclictest32759-21grep22:45:221
1316099187,6cyclictest52950irq/141-enp3s0-22:25:251
13160991813,3cyclictest20879-21sh00:36:131
13160991812,4cyclictest14167-21sh00:07:371
13160991811,5cyclictest52950irq/141-enp3s0-00:10:171
13160991810,6cyclictest30827-21fw_packets21:05:141
13160991810,6cyclictest20108-21cut20:40:141
13160991810,6cyclictest14295-21which19:10:221
13160991810,5cyclictest32263-21diskmemload23:49:021
1315999189,7cyclictest22758-21cut20:45:220
1315999188,8cyclictest28820-21rm23:06:140
1315999182,11cyclictest11050-21apt-get22:30:130
13159991811,5cyclictest15248-21date19:15:010
13159991810,6cyclictest18337-21apt00:35:100
13159991810,6cyclictest16674-21cat22:10:140
13160991712,3cyclictest23905-21/usr/sbin/munin20:50:241
1315999178,7cyclictest24505-21sed20:50:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional