You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-17 - 10:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Wed Jun 17, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2000829481,9sleep00-21swapper/019:07:510
1993229380,9sleep20-21swapper/219:06:512
1987628976,9sleep30-21swapper/319:06:083
1982528968,16sleep10-21swapper/119:05:301
246082770,0sleep20-21swapper/223:48:572
17132620,0sleep10-21swapper/122:00:011
266052320,0sleep3381rcuc/322:14:423
2026199290,28cyclictest0-21swapper/121:26:151
2026599280,0cyclictest0-21swapper/200:22:122
2026599280,0cyclictest0-21swapper/200:22:122
2026199280,26cyclictest0-21swapper/121:55:121
20267992519,2cyclictest225-21systemd-journal00:08:383
2026599250,22cyclictest0-21swapper/223:00:282
2026599250,22cyclictest0-21swapper/221:55:212
20261992522,2cyclictest25-21ksoftirqd/119:30:161
2026599240,1cyclictest0-21swapper/220:05:162
20261992422,2cyclictest0-21swapper/100:07:001
20265992322,1cyclictest0-21swapper/222:29:432
20265992322,1cyclictest0-21swapper/222:29:422
2026599230,23cyclictest0-21swapper/223:44:552
20267992221,1cyclictest0-21swapper/321:57:333
2026599210,21cyclictest0-21swapper/223:32:122
2026599210,20cyclictest0-21swapper/222:37:022
2026799202,14cyclictest0-21swapper/321:45:003
2026599202,11cyclictest32188-21/usr/sbin/munin21:20:162
20265992012,5cyclictest6401-21missed_timers22:40:212
20261992014,2cyclictest225-21systemd-journal23:41:531
2026199200,19cyclictest0-21swapper/123:18:061
20265991911,7cyclictest32-21ksoftirqd/219:10:132
2026599190,18cyclictest0-21swapper/223:20:272
20261991914,3cyclictest0-21swapper/100:26:361
2025699190,1cyclictest0-21swapper/000:22:040
2025699190,1cyclictest0-21swapper/000:22:030
20267991816,1cyclictest0-21swapper/323:40:373
2026799181,16cyclictest0-21swapper/300:30:093
2026599181,16cyclictest0-21swapper/222:01:562
20261991816,2cyclictest0-21swapper/122:57:221
2026199180,3cyclictest14432-21ssh23:23:541
2026799170,1cyclictest0-21swapper/323:52:353
2026799170,1cyclictest0-21swapper/300:01:053
20261991714,2cyclictest241rcuc/100:15:271
20261991712,5cyclictest38150irq/131-snd_hda22:48:471
2026199170,1cyclictest0-21swapper/120:30:001
2026199170,15cyclictest0-21swapper/123:47:341
2026199170,0cyclictest0-21swapper/121:11:131
2025699176,5cyclictest4476-21apt-get19:45:000
2025699170,16cyclictest0-21swapper/023:29:510
2025699170,16cyclictest0-21swapper/019:35:130
2025699170,0cyclictest0-21swapper/023:55:370
2025699170,0cyclictest0-21swapper/023:40:120
20267991614,2cyclictest0-21swapper/322:40:083
2026599160,14cyclictest0-21swapper/200:14:302
2026599160,13cyclictest0-21swapper/200:04:092
20256991610,2cyclictest225-21systemd-journal22:37:350
20267991513,1cyclictest14125-21rm21:47:583
2026799150,2cyclictest225-21systemd-journal00:00:003
2026799150,1cyclictest0-21swapper/323:35:033
2026599151,2cyclictest0-21swapper/221:25:142
20265991511,3cyclictest6525-21wc19:45:262
2026599151,0cyclictest0-21swapper/220:40:142
2026199155,5cyclictest225-21systemd-journal21:17:001
2026199153,12cyclictest13640-21diskmemload22:05:321
20261991515,0cyclictest0-21swapper/121:33:221
20261991513,1cyclictest25-21ksoftirqd/100:14:501
2026199150,14cyclictest13640-21diskmemload21:37:121
2026199150,14cyclictest0-21swapper/123:13:511
2026199150,0cyclictest141rcu_preempt22:12:051
2025699150,1cyclictest0-21swapper/021:16:260
17072150,0chrt0-21swapper/200:32:252
2026799148,3cyclictest0-21swapper/323:25:133
2026799142,8cyclictest8609-21/usr/sbin/munin23:20:163
20267991413,1cyclictest0-21swapper/323:00:273
20267991411,2cyclictest475-21in:imuxsock21:21:253
20267991411,2cyclictest21005-21tr22:30:163
20267991411,2cyclictest21005-21tr22:30:153
20267991411,2cyclictest16757-21cat23:05:273
2026799140,1cyclictest0-21swapper/321:38:143
2026799140,13cyclictest0-21swapper/300:22:183
2026799140,13cyclictest0-21swapper/300:22:183
2026599148,5cyclictest516-21gdbus19:15:222
2026599147,6cyclictest10430-21cron00:00:002
2026599143,2cyclictest4161-21apache_volume00:15:132
2026599142,1cyclictest13640-21diskmemload21:10:412
20265991410,3cyclictest5519-21grep22:20:262
20265991410,3cyclictest27574-21wc23:50:192
20265991410,3cyclictest22481-21awk00:25:232
20265991410,2cyclictest29195-21perf20:35:002
20265991410,2cyclictest12209-21cat20:00:002
2026599140,13cyclictest0-21swapper/221:31:152
2026199149,4cyclictest3913-21wc22:20:141
2026199149,1cyclictest225-21systemd-journal22:51:431
2026199148,5cyclictest0-21swapper/123:26:571
2026199148,2cyclictest225-21systemd-journal22:41:411
2026199148,2cyclictest225-21systemd-journal22:36:391
20261991411,2cyclictest10650-21cat00:00:001
2026199140,14cyclictest0-21swapper/123:54:091
20256991413,1cyclictest0-21swapper/023:20:360
20256991413,1cyclictest0-21swapper/021:30:140
20256991411,2cyclictest376-21tr21:20:200
20256991410,3cyclictest5567-21head00:15:220
2026799139,3cyclictest31100-21sed22:35:273
2026799138,4cyclictest29152-21wc00:30:013
20267991310,2cyclictest4154-21grep22:20:153
20267991310,2cyclictest3987-21grep22:00:293
20267991310,2cyclictest14821-21cat22:45:203
2026799130,1cyclictest0-21swapper/323:30:233
2026799130,12cyclictest5710-21grep19:45:173
2026599139,3cyclictest1064-21grep19:35:192
20265991311,1cyclictest0-21swapper/221:38:392
20265991311,1cyclictest0-21swapper/200:08:352
20265991310,2cyclictest4483-21grep20:50:142
20265991310,2cyclictest3091-21tail19:40:172
20265991310,2cyclictest25462-21grep20:25:172
20265991310,2cyclictest24454-21awk20:25:012
20265991310,2cyclictest11288-21grep22:05:192
2026599130,12cyclictest2083-21basename20:45:142
2026599130,12cyclictest2083-21basename20:45:132
2026199132,10cyclictest6721-21missed_timers00:35:191
2026199131,1cyclictest141rcu_preempt23:04:391
20261991310,2cyclictest6599-21wc20:55:001
20261991310,2cyclictest21055-21cut22:30:161
20261991310,2cyclictest21055-21cut22:30:151
20261991310,2cyclictest17269-21cron20:10:011
2025699139,3cyclictest4397-21head22:20:170
2025699139,2cyclictest31764-21sh20:40:010
2025699139,2cyclictest23520-21grep19:15:170
2025699132,10cyclictest14395-21sh22:06:580
20256991310,2cyclictest4108-21cron20:50:000
20256991310,2cyclictest4108-21cron20:50:000
20256991310,2cyclictest29602-21basename00:30:140
20256991310,2cyclictest27389-21ssh23:50:170
20256991310,2cyclictest10782-21sed21:45:260
2025699130,1cyclictest0-21swapper/000:02:270
2025699130,12cyclictest0-21swapper/022:54:410
2025699130,0cyclictest0-21swapper/021:38:380
2026799129,2cyclictest6027-21sed20:50:263
2026799129,2cyclictest29293-21sed19:25:283
2026799129,2cyclictest25235-21ls23:10:263
2026799129,2cyclictest14999-21tail21:10:163
2026799121,10cyclictest28051-21/usr/sbin/munin00:10:133
20267991210,1cyclictest25324-21grep20:25:163
2026799120,6cyclictest0-21swapper/321:15:183
2026799120,1cyclictest0-21swapper/300:35:423
2026599129,2cyclictest28402-21grep22:15:192
2026599129,2cyclictest25936-21cut19:20:142
2026599129,2cyclictest22931-21grep22:50:182
2026599129,2cyclictest18308-21grep20:10:172
2026599128,3cyclictest22631-21ssh23:28:322
2026599128,2cyclictest19892-21tail22:10:142
2026599128,2cyclictest0-21swapper/221:50:142
2026599122,10cyclictest0-21swapper/221:48:492
2026599120,3cyclictest0-21swapper/222:56:222
2026599120,1cyclictest0-21swapper/200:38:172
2026599120,11cyclictest32714-21grep23:15:162
2026599120,0cyclictest0-21swapper/223:35:172
2026199129,2cyclictest519-21ssh21:20:231
2026199129,2cyclictest30167-21ssh00:30:161
2026199129,2cyclictest14586-21head00:20:281
2026199129,2cyclictest14586-21head00:20:271
2026199129,2cyclictest12087-21ls00:00:181
2026199129,2cyclictest11525-21grep19:55:271
2026199123,8cyclictest8531-21sort21:45:001
2026199120,11cyclictest0-21swapper/119:45:141
2026199120,0cyclictest0-21swapper/121:51:481
2025699129,2cyclictest8289-21grep21:25:170
2025699129,2cyclictest373-21grep20:40:180
2025699129,2cyclictest32028-21tr23:15:130
2025699129,2cyclictest29199-21basename20:35:000
2025699129,2cyclictest25210-21grep23:30:130
2025699129,2cyclictest21799-21cut00:25:170
2025699129,2cyclictest13950-21grep22:25:280
2025699129,2cyclictest13950-21grep22:25:270
2025699129,2cyclictest13645-21munin-run21:10:000
2025699121,1cyclictest0-21swapper/022:10:110
20256991210,1cyclictest0-21swapper/021:40:120
2025699120,2cyclictest0-21swapper/023:10:280
2026799119,1cyclictest4191-21sh00:15:153
2026799119,1cyclictest4110-21sh20:50:013
2026799119,1cyclictest4110-21sh20:50:003
2026799118,2cyclictest8724-21chrt20:58:233
2026799118,2cyclictest710-21grep20:40:213
2026799118,2cyclictest385-21expr19:35:143
2026799118,2cyclictest30743-21grep19:30:163
2026799118,2cyclictest30175-21sed20:35:153
2026799118,2cyclictest29030-21grep22:15:243
2026799118,2cyclictest26928-21grep19:20:233
2026799118,2cyclictest26072-21ssh22:51:563
2026799118,2cyclictest19668-21ssh21:50:403
2026799118,2cyclictest18752-21ssh22:29:043
2026799118,2cyclictest18752-21ssh22:29:033
2026799118,2cyclictest17994-21ssh21:30:273
2026799118,2cyclictest17876-21chrt23:45:053
2026799118,2cyclictest15796-21grep20:05:193
2026799117,3cyclictest31661-21ssh23:15:023
2026799117,3cyclictest21596-21strings19:10:223
2026799117,3cyclictest11687-21cut22:05:223
2026799111,2cyclictest31081-21df22:55:163
2026599118,2cyclictest3578-21ssh21:41:192
2026599118,2cyclictest32146-21basename19:34:592
2026599118,2cyclictest28135-21df19:25:172
2026599118,2cyclictest25455-21sed23:10:272
2026599118,2cyclictest21501-21cut22:30:182
2026599118,2cyclictest21501-21cut22:30:172
2026599118,2cyclictest15850-21sed23:05:182
2026599118,2cyclictest15031-21cut22:45:232
2026599116,5cyclictest10772-21kworker/u8:221:15:352
2026599110,10cyclictest9509-21taskset19:53:212
2026199118,2cyclictest8893-21sleep21:00:011
2026199118,2cyclictest32195-21ssh22:17:301
2026199118,2cyclictest310-21grep20:40:181
2026199118,2cyclictest30013-21ssh23:32:421
2026199118,2cyclictest26525-21wc19:20:191
2026199118,2cyclictest2400-21grep19:40:141
2026199118,2cyclictest2263-21tr20:45:141
2026199118,2cyclictest2263-21tr20:45:131
2026199118,2cyclictest19657-21ssh23:07:271
2026199118,2cyclictest18886-21tr20:10:231
2026199118,2cyclictest1681-21sed19:35:261
2026199118,2cyclictest13265-21cut22:25:201
2026199118,2cyclictest13265-21cut22:25:201
2026199118,2cyclictest12593-21wget20:00:141
2025699119,1cyclictest28411-21head22:15:200
2025699118,2cyclictest8394-21ssh00:36:040
2025699118,2cyclictest8314-21ssh22:41:170
2025699118,2cyclictest8007-21grep23:00:240
2025699118,2cyclictest5377-21cat20:50:190
2025699118,2cyclictest32119-21cron19:34:590
2025699118,2cyclictest30466-21ssh00:10:410
2025699118,2cyclictest2981-21sed23:35:260
2025699118,2cyclictest27389-21ls19:25:000
2025699118,2cyclictest26872-21grep21:55:160
2025699118,2cyclictest22729-21grep22:30:270
2025699118,2cyclictest22729-21grep22:30:270
2025699118,2cyclictest2176-21grep22:00:150
2025699118,2cyclictest20450-21chrt19:10:020
2025699118,2cyclictest19811-21cat20:15:010
2025699118,2cyclictest18653-21tr21:50:210
2025699118,2cyclictest18240-21sh21:12:000
2025699118,2cyclictest17884-21ssh23:45:050
2025699118,2cyclictest15623-21sh23:05:170
2025699118,2cyclictest13906-21tr22:45:160
2025699117,3cyclictest32373-21ls22:55:260
2025699117,3cyclictest23767-21wc20:20:260
2025699116,2cyclictest452-21systemd-logind00:08:380
2025699110,10cyclictest6266-21gzip19:45:220
2026799108,1cyclictest7513-21basename19:50:113
2026799108,1cyclictest7411-21ssh21:25:053
2026799108,1cyclictest28490-21expr20:30:263
2026799108,1cyclictest23310-21grep20:20:193
2026799108,1cyclictest22877-21apt-get19:15:133
2026799108,1cyclictest218872sleep320:18:123
2026799108,1cyclictest12983-21cat21:05:273
2026799108,1cyclictest10695-21expr19:55:173
2026799107,2cyclictest3098-21head19:40:163
2026799107,2cyclictest17896-21expr20:10:153
2026799107,2cyclictest10944-21chrt21:01:083
2026799100,9cyclictest12397-21sed20:00:003
2026599108,1cyclictest6971-21wc20:55:162
2026599108,1cyclictest31650-21chrt20:38:392
2026599108,1cyclictest10504-21which21:00:212
2026599107,3cyclictest32-21ksoftirqd/220:15:242
2026599107,2cyclictest13280-21sed20:00:172
2026599107,2cyclictest12052-21df_abs21:05:162
2026199108,1cyclictest28319-21cut19:25:191
2026199108,1cyclictest27413-21expr20:30:141
2026199108,1cyclictest23454-21ntpq20:20:211
2026199108,1cyclictest1174-21cut23:35:131
2026199107,2cyclictest9881-21uname21:45:181
2026199107,2cyclictest8465-21uname19:50:171
2026199107,2cyclictest21130-21cut19:10:171
2026199107,2cyclictest20035-21basename20:15:131
2026199106,2cyclictest29695-21grep20:35:151
2026199104,4cyclictest11023-21chrt21:02:061
2026199100,9cyclictest11914-21expr21:05:151
2026199100,3cyclictest225-21systemd-journal19:15:571
2025699108,1cyclictest9581-21cat21:00:120
2025699108,1cyclictest17216-21chrt20:09:200
2025699107,2cyclictest9509-21chrt19:53:210
2025699107,2cyclictest28113-21expr19:25:160
2025699107,2cyclictest14795-21date20:05:000
2025699107,2cyclictest12047-21perf20:00:010
2025699106,3cyclictest8854-21cron21:00:010
202569997,1cyclictest26213-21cat20:25:250
202569997,1cyclictest19881-21taskset20:15:060
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional