You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 23:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Thu Jul 30, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26952993433,1cyclictest16399-21lspci09:02:380
26954993231,1cyclictest0-21swapper/109:02:381
2695899290,29cyclictest0-21swapper/209:02:382
26962992827,1cyclictest16545-21lspci09:02:383
26962992523,1cyclictest23290-21awk11:35:013
26962992522,2cyclictest607-21in:imuxsock12:30:003
26954992523,1cyclictest0-21swapper/112:21:251
2695499250,24cyclictest21254-21diskmemload10:48:421
2695499250,1cyclictest26432-21sh10:12:001
26962992423,1cyclictest4842-21ssh12:16:213
26962992422,1cyclictest21254-21diskmemload12:04:063
26962992421,2cyclictest14805-21grep10:58:353
2696299241,2cyclictest25121-21vmstat12:33:393
2696299240,23cyclictest21254-21diskmemload10:27:573
2696299240,23cyclictest0-21swapper/309:56:273
2696299240,1cyclictest18127-21ssh09:35:193
2696299240,1cyclictest0-21swapper/311:58:493
2696299240,0cyclictest0-21swapper/309:43:073
26958992419,4cyclictest0-21swapper/212:12:432
2695899240,2cyclictest12205-21ssh10:57:152
2695899240,24cyclictest0-21swapper/209:23:572
2695899240,1cyclictest10691-21rm09:28:432
2695899240,0cyclictest21254-21diskmemload12:00:222
2695899240,0cyclictest21254-21diskmemload09:18:192
26954992423,1cyclictest21254-21diskmemload12:14:331
26954992423,1cyclictest0-21swapper/109:55:451
2695499241,22cyclictest211rcuc/109:25:001
2695499240,24cyclictest0-21swapper/111:35:131
2695499240,24cyclictest0-21swapper/110:04:331
2695499240,24cyclictest0-21swapper/109:30:071
2695499240,23cyclictest21254-21diskmemload12:27:081
2695499240,23cyclictest21254-21diskmemload11:13:001
2695499240,23cyclictest16801-21ssh11:00:151
2695499240,23cyclictest14757-21idleruntime-cro10:30:011
2695499240,0cyclictest21254-21diskmemload12:07:511
2695499240,0cyclictest21254-21diskmemload12:07:511
2695499240,0cyclictest0-21swapper/111:20:401
2695299241,22cyclictest619-21polkitd09:45:010
2695299240,23cyclictest2088-21ssh11:16:300
2695299240,23cyclictest18571-21sed12:28:310
2695299240,23cyclictest0-21swapper/009:12:360
2695299240,1cyclictest1826-21ssh09:21:400
2695299240,1cyclictest0-21swapper/010:30:550
26962992323,0cyclictest0-21swapper/311:42:293
26962992323,0cyclictest0-21swapper/311:26:433
26962992323,0cyclictest0-21swapper/310:45:103
26962992323,0cyclictest0-21swapper/309:16:473
26962992322,1cyclictest0-21swapper/310:31:543
26962992322,1cyclictest0-21swapper/309:20:133
26962992321,1cyclictest21254-21diskmemload11:03:363
26962992321,1cyclictest0-21swapper/310:39:013
26962992320,2cyclictest606-21dbus-daemon08:10:013
26962992320,2cyclictest21254-21diskmemload12:07:033
26962992320,2cyclictest21254-21diskmemload12:07:033
2696299230,23cyclictest21254-21diskmemload11:08:013
2696299230,23cyclictest0-21swapper/312:13:493
2696299230,23cyclictest0-21swapper/310:50:083
2696299230,22cyclictest8344-21ssh11:50:253
2696299230,22cyclictest6539-21rm09:25:303
2696299230,22cyclictest3988-21apt08:38:283
2696299230,22cyclictest30831-21ssh10:43:543
2696299230,22cyclictest2794-21sh11:45:373
2696299230,22cyclictest21852-21ssh09:10:263
2696299230,22cyclictest21254-21diskmemload10:18:133
2696299230,22cyclictest21254-21diskmemload09:47:093
2696299230,22cyclictest0-21swapper/309:32:273
2696299230,1cyclictest0-21swapper/310:10:093
2696299230,1cyclictest0-21swapper/310:02:033
2696299230,0cyclictest0-21swapper/312:22:033
26958992323,0cyclictest0-21swapper/209:52:112
26958992322,1cyclictest9611-21ssh10:53:572
26958992320,2cyclictest12509-21ssh11:25:102
2695899230,2cyclictest2996-21ssh11:17:472
2695899230,23cyclictest21254-21diskmemload09:46:342
2695899230,23cyclictest0-21swapper/212:39:212
2695899230,23cyclictest0-21swapper/212:30:202
2695899230,22cyclictest24465-21ssh11:36:162
2695899230,22cyclictest21254-21diskmemload10:38:252
2695899230,22cyclictest19022-21ssh09:36:372
2695899230,22cyclictest15148-21ssh12:25:032
2695899230,22cyclictest10257-21ssh11:53:102
2695899230,22cyclictest0-21swapper/211:20:272
26954992323,0cyclictest0-21swapper/112:36:061
26954992323,0cyclictest0-21swapper/112:30:311
26954992323,0cyclictest0-21swapper/110:35:211
26954992323,0cyclictest0-21swapper/110:09:581
26954992322,1cyclictest24924-21ssh09:41:531
26954992321,1cyclictest16857-21ssh11:28:551
26954992321,1cyclictest15417-21ssh10:30:421
26954992321,1cyclictest0-21swapper/111:44:381
26954992321,1cyclictest0-21swapper/111:19:211
26954992320,2cyclictest9094-21sh11:51:301
26954992320,2cyclictest15305-21rm11:56:501
2695499230,2cyclictest4190-21ssh12:15:221
2695499230,23cyclictest0-21swapper/110:22:571
2695499230,22cyclictest0-21swapper/111:48:451
2695499230,1cyclictest10387-21sh10:54:561
2695499230,0cyclictest21254-21diskmemload11:09:351
2695499230,0cyclictest0-21swapper/110:18:241
2695499230,0cyclictest0-21swapper/109:48:581
26952992323,0cyclictest0-21swapper/011:55:260
26952992323,0cyclictest0-21swapper/011:45:100
2695299232,21cyclictest62550irq/127-eno111:35:250
26952992322,1cyclictest0-21swapper/012:15:500
26952992322,1cyclictest0-21swapper/010:48:110
26952992321,1cyclictest9455-21sh07:40:000
26952992321,1cyclictest7677-21ssh11:21:200
26952992321,1cyclictest29777-21sh09:45:560
2695299230,2cyclictest28702-21idleruntime-cro08:20:010
2695299230,2cyclictest0-21swapper/010:15:150
2695299230,23cyclictest21254-21diskmemload10:11:560
2695299230,23cyclictest0-21swapper/012:38:550
2695299230,23cyclictest0-21swapper/011:25:390
2695299230,23cyclictest0-21swapper/011:02:530
2695299230,23cyclictest0-21swapper/010:25:240
2695299230,22cyclictest9452-21ssh11:51:570
2695299230,22cyclictest5162-21sh10:50:120
2695299230,22cyclictest0-21swapper/012:01:250
2695299230,22cyclictest0-21swapper/011:40:460
2695299230,1cyclictest8397-21ssh10:23:580
2695299230,1cyclictest27235-21sh11:09:280
2695299230,1cyclictest0-21swapper/012:32:520
2695299230,1cyclictest0-21swapper/009:50:290
2695299230,0cyclictest0-21swapper/012:08:170
2695299230,0cyclictest0-21swapper/012:08:170
2695299230,0cyclictest0-21swapper/011:11:280
2695299230,0cyclictest0-21swapper/009:59:250
2695299230,0cyclictest0-21swapper/009:26:410
26962992221,0cyclictest0-21swapper/307:28:273
26962992220,1cyclictest21008-21sh10:07:133
26962992220,1cyclictest1589-21apt-get08:33:293
26962992220,1cyclictest0-21swapper/311:35:203
26962992220,1cyclictest0-21swapper/311:20:243
26962992220,1cyclictest0-21swapper/309:50:393
2696299220,1cyclictest4219-21sh10:20:383
2696299220,1cyclictest21254-21diskmemload12:37:473
2696299220,0cyclictest21254-21diskmemload11:19:433
26958992222,0cyclictest0-21swapper/212:15:232
26958992222,0cyclictest0-21swapper/212:05:272
26958992222,0cyclictest0-21swapper/212:05:272
26958992222,0cyclictest0-21swapper/210:22:452
26958992222,0cyclictest0-21swapper/209:40:482
26958992220,1cyclictest24391-21apt-get08:13:282
26958992220,1cyclictest13133-21munin-run08:55:002
26958992220,1cyclictest0-21swapper/211:05:022
2695899220,22cyclictest20589-21ssh10:06:412
2695899220,22cyclictest10324-21sh10:26:132
2695899220,21cyclictest4837-21latency_hist10:50:012
2695899220,21cyclictest25239-21sh10:10:182
2695899220,21cyclictest10239-21rm12:20:572
2695899220,21cyclictest0-21swapper/211:30:542
2695899220,21cyclictest0-21swapper/210:40:392
2695899220,21cyclictest0-21swapper/210:15:022
2695899220,21cyclictest0-21swapper/209:10:062
2695899220,1cyclictest9508-21rm09:57:142
2695899220,1cyclictest31098-21awk08:25:012
2695899220,1cyclictest29792-21ssh11:40:542
2695899220,1cyclictest29088-21ssh11:11:442
2695899220,1cyclictest0-21swapper/211:01:012
2695899220,1cyclictest0-21swapper/210:04:012
2695899220,0cyclictest0-21swapper/211:45:372
2695899220,0cyclictest0-21swapper/207:18:282
26954992222,0cyclictest0-21swapper/109:50:441
26954992222,0cyclictest0-21swapper/109:29:461
26954992220,1cyclictest14260-21grep07:50:001
2695499220,21cyclictest0-21swapper/109:35:011
2695499220,21cyclictest0-21swapper/109:15:171
2695499220,1cyclictest19340-21ssh11:32:011
2695499220,1cyclictest12513-21sh10:57:391
26952992222,0cyclictest21254-21diskmemload10:08:000
26952992222,0cyclictest0-21swapper/010:40:210
26952992219,2cyclictest313-21systemd-journal08:35:000
2695299220,22cyclictest0-21swapper/010:56:070
2695299220,21cyclictest8347-21munin-plugin-st08:45:000
2695299220,21cyclictest0-21swapper/011:32:250
2695299220,21cyclictest0-21swapper/009:35:310
2695299220,21cyclictest0-21swapper/009:30:020
2695299220,21cyclictest0-21swapper/009:16:520
2695299220,21cyclictest0-21swapper/008:03:280
2695299220,1cyclictest20891-21ssh10:35:270
2695299220,1cyclictest14013-21ssh10:00:150
2695299220,0cyclictest0-21swapper/012:21:000
26962992121,0cyclictest19402-21apt-get09:08:283
26962992119,1cyclictest19601-21apt-get08:03:283
2696299210,20cyclictest29840-21apt-get07:18:283
2696299210,1cyclictest32253-21apt-get07:23:273
2696299210,1cyclictest0-21swapper/307:35:013
2696299210,0cyclictest0-21swapper/307:48:283
26958992121,0cyclictest0-21swapper/208:48:312
2695899210,21cyclictest0-21swapper/211:55:052
2695899210,20cyclictest0-21swapper/208:18:282
26954992121,0cyclictest0-21swapper/108:35:001
26954992120,0cyclictest27441-21apt-get07:13:271
2695499210,21cyclictest0-21swapper/108:03:281
26952992121,0cyclictest0-21swapper/008:38:280
26952992121,0cyclictest0-21swapper/008:28:260
2695299210,21cyclictest0-21swapper/007:13:270
2695299210,20cyclictest21276-21wc09:10:010
2695299210,0cyclictest0-21swapper/007:33:280
26962992020,0cyclictest0-21swapper/308:23:283
26958992019,0cyclictest0-21swapper/208:33:292
2695899200,20cyclictest0-21swapper/207:48:282
2695899200,1cyclictest21480-21perf08:05:002
2695899200,0cyclictest0-21swapper/208:58:302
26954992020,0cyclictest0-21swapper/108:58:291
26954992020,0cyclictest0-21swapper/107:58:281
26952992020,0cyclictest0-21swapper/008:13:280
2695299200,0cyclictest0-21swapper/007:18:280
26962991919,0cyclictest0-21swapper/308:28:263
26962991919,0cyclictest0-21swapper/307:38:283
26962991917,1cyclictest16667-21wc07:55:003
26958991919,0cyclictest0-21swapper/207:58:292
26954991919,0cyclictest0-21swapper/107:33:281
26954991919,0cyclictest0-21swapper/107:28:271
26954991918,0cyclictest10005-21apt-get07:43:281
26952991919,0cyclictest23876-21perf08:10:010
26952991919,0cyclictest0-21swapper/007:48:280
26952991918,0cyclictest13647-21apt-get08:58:290
26962991818,0cyclictest0-21swapper/308:53:303
26962991818,0cyclictest0-21swapper/308:43:293
26958991818,0cyclictest7431-21apt-get07:38:282
26958991818,0cyclictest0-21swapper/208:08:282
26958991818,0cyclictest0-21swapper/207:43:282
2695899180,18cyclictest0-21swapper/209:08:282
26954991818,0cyclictest0-21swapper/109:08:281
26954991817,0cyclictest6409-21apt-get08:43:291
26952991818,0cyclictest0-21swapper/008:48:310
26952991818,0cyclictest0-21swapper/007:43:280
26952991816,1cyclictest2636-21apt-get07:28:270
26958991717,0cyclictest0-21swapper/208:43:292
26958991717,0cyclictest0-21swapper/207:23:272
26958991715,1cyclictest609-21rs:main2
26954991717,0cyclictest0-21swapper/108:53:301
26954991717,0cyclictest0-21swapper/108:23:281
26954991717,0cyclictest0-21swapper/107:38:281
2695499170,0cyclictest3989-21apt-get08:38:281
26952991716,1cyclictest17186-21apt-get07:58:280
26952991716,1cyclictest11255-21apt-get08:53:300
26962991616,0cyclictest0-21swapper/308:58:303
26962991616,0cyclictest0-21swapper/307:43:283
26954991616,0cyclictest0-21swapper/108:18:281
26954991616,0cyclictest0-21swapper/108:08:281
26952991614,1cyclictest16675-21munin-run07:55:000
2696299150,1cyclictest27698-21memory08:18:323
2696299150,1cyclictest17988-21irqstats07:58:323
26958991515,0cyclictest0-21swapper/208:38:282
26958991513,1cyclictest27906-21/usr/sbin/munin07:13:292
26954991514,1cyclictest32253-21apt-get07:23:271
26952991515,0cyclictest29213-21apt-get08:23:280
26952991515,0cyclictest0-21swapper/007:23:270
26962991414,0cyclictest0-21swapper/308:11:153
2696299140,14cyclictest0-21swapper/308:45:073
2696299140,14cyclictest0-21swapper/307:10:313
26958991414,0cyclictest0-21swapper/208:25:102
2695899140,0cyclictest0-21swapper/207:50:192
2695899140,0cyclictest0-21swapper/207:25:242
26954991414,0cyclictest0-21swapper/107:50:481
2695499140,14cyclictest0-21swapper/108:45:011
2695499140,14cyclictest0-21swapper/108:10:101
2695499140,0cyclictest0-21swapper/108:25:301
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional