You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-30 - 16:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Thu Apr 30, 2026 12:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1788669910224,43cyclictest275-21dbus-daemon10:14:400
1788789910110,54cyclictest223095-21ssh10:35:012
178866998817,36cyclictest189447-21sendmail09:20:010
178866998525,35cyclictest0-21swapper/010:38:360
178866998512,32cyclictest0-21swapper/010:03:520
178866998424,17cyclictest184588-21munin-run09:00:000
178866998330,36cyclictest239214-21ssh11:12:100
178866998224,20cyclictest200329-21sed09:45:010
178866998025,14cyclictest270666-21cut12:25:020
178866998025,14cyclictest270666-21cut12:25:020
178866998014,36cyclictest0-21swapper/011:18:570
178866997926,22cyclictest251301-21sed11:40:030
17888199778,11cyclictest37-21ksoftirqd/310:35:013
178866997727,27cyclictest0-21swapper/010:48:530
178866997524,14cyclictest243984-21ssh11:23:200
178866997523,19cyclictest227408-21date10:45:020
178866997521,49cyclictest190606-21ssh09:22:350
17886699750,37cyclictest214447-21modprobe10:15:150
178866997427,16cyclictest253309-21munin-run11:45:010
178866997424,14cyclictest205900-21ssh09:58:120
178866997415,32cyclictest0-21swapper/011:52:180
178866997324,12cyclictest0-21swapper/007:15:000
178866997320,20cyclictest266790-21ssh12:15:480
178866997224,15cyclictest259466-21ssh11:59:090
178866997223,13cyclictest185111-21diskmemload12:34:390
178866997220,21cyclictest197031-21sh09:37:340
178866997129,12cyclictest237390-21ssh11:08:020
178866997120,12cyclictest254906-21ssh11:48:290
178866997063,5cyclictest12-21ksoftirqd/011:25:030
178866997033,4cyclictest182444-21munin-run08:20:000
178866997026,13cyclictest185111-21diskmemload10:58:430
178866997024,15cyclictest204457-21munin-run09:55:000
178866996932,5cyclictest219089-21ssh10:25:360
178866996930,35cyclictest0-21swapper/009:10:020
178866996922,15cyclictest247306-21ssh11:30:410
178866996922,15cyclictest182727-21systemd-run08:25:010
178866996922,14cyclictest223059-21latency_hist10:35:020
178881996816,13cyclictest37-21ksoftirqd/310:57:323
178874996811,14cyclictest131rcu_preempt10:40:031
178866996826,11cyclictest262015-21munin-run12:05:020
178866996821,13cyclictest235106-21ssh11:02:370
178866996818,17cyclictest200802-21ssh09:46:030
178878996740,7cyclictest227087-21kworker/2:010:59:382
17886699670,35cyclictest276076-21ssh12:37:310
17887899652,5cyclictest131rcu_preempt09:20:032
178866996524,15cyclictest264342-21sed12:10:020
178866996460,2cyclictest0-21swapper/009:25:020
178866996422,37cyclictest217353-21ssh10:21:430
178878996339,5cyclictest131rcu_preempt10:25:022
17886699637,20cyclictest275-21dbus-daemon08:15:010
178866996349,11cyclictest187082-21sshd09:14:430
178866996345,15cyclictest3467602-21systemd-journal12:27:280
178866996328,31cyclictest0-21swapper/007:35:020
178866996328,31cyclictest0-21swapper/007:35:020
17888199616,9cyclictest37-21ksoftirqd/307:15:013
17886699613,8cyclictest210916-21ssh10:09:500
178881996051,6cyclictest361rcuc/308:36:543
178866996031,25cyclictest0-21swapper/010:50:460
178866996029,27cyclictest181394-21latency_hist08:00:030
178866996027,29cyclictest183190-21sshd08:30:430
17886699601,41cyclictest0-21swapper/007:40:460
17887899595,29cyclictest227191-21kworker/u8:111:20:012
178866995945,11cyclictest183447-21sshd08:36:520
178866995920,35cyclictest0-21swapper/009:30:550
178881995810,8cyclictest37-21ksoftirqd/309:40:013
178866995845,10cyclictest184286-21sshd08:50:280
178866995844,10cyclictest179576-21sshd07:21:450
17888199571,22cyclictest0-21swapper/311:57:513
178866995727,26cyclictest0-21swapper/008:26:330
178866995724,5cyclictest265621-21ssh12:13:210
17888199567,10cyclictest37-21ksoftirqd/309:40:153
178866995642,9cyclictest184827-21sshd09:01:440
178866995617,35cyclictest0-21swapper/007:29:440
17887899555,35cyclictest180074-21kworker/u8:007:35:032
17887899555,35cyclictest180074-21kworker/u8:007:35:022
178878995546,6cyclictest211010-21kworker/2:310:54:122
178874995537,14cyclictest1-21systemd12:00:011
178866995536,14cyclictest364-21snmpd07:53:060
178866995535,16cyclictest183830-21munin-run08:45:010
178866995513,13cyclictest181887-21sshd08:05:170
178881995434,15cyclictest1-21systemd08:50:013
178878995444,6cyclictest181447-21kworker/2:210:15:152
178878995435,14cyclictest1-21systemd09:05:012
178866995446,4cyclictest181433-21munin-run08:00:020
178866995432,18cyclictest184072-21munin-run08:50:010
178866995416,32cyclictest180488-21cat07:40:010
17888199533,9cyclictest37-21ksoftirqd/312:08:253
178881995335,14cyclictest275-21dbus-daemon12:20:013
178881995332,14cyclictest275-21dbus-daemon10:14:363
17888199533,11cyclictest37-21ksoftirqd/312:37:463
17888199531,21cyclictest0-21swapper/311:54:443
178878995343,6cyclictest181447-21kworker/2:209:35:012
178866995342,8cyclictest48-21kcompactd007:48:440
178866995335,13cyclictest364-21snmpd07:15:390
17888199526,7cyclictest37-21ksoftirqd/308:00:013
17888199524,7cyclictest37-21ksoftirqd/310:15:153
178881995231,16cyclictest3467602-21systemd-journal09:20:033
17888199522,12cyclictest37-21ksoftirqd/310:37:543
17888199521,8cyclictest37-21ksoftirqd/311:47:163
178881995213,6cyclictest131rcu_preempt11:12:173
17888199521,21cyclictest0-21swapper/309:28:163
178881995210,17cyclictest37-21ksoftirqd/312:00:573
178878995243,6cyclictest211010-21kworker/2:310:48:142
178878995243,5cyclictest257649-21kworker/2:211:58:172
178878995243,5cyclictest181447-21kworker/2:208:29:052
178878995242,6cyclictest181447-21kworker/2:209:41:442
17888199519,5cyclictest131rcu_preempt10:53:283
178881995140,8cyclictest361rcuc/310:47:103
178881995116,22cyclictest361rcuc/311:03:453
17888199511,20cyclictest0-21swapper/311:18:343
17888199511,20cyclictest0-21swapper/309:53:443
17888199511,20cyclictest0-21swapper/307:40:003
178881995111,5cyclictest131rcu_preempt10:43:343
178881995111,5cyclictest131rcu_preempt09:11:373
178878995143,5cyclictest211010-21kworker/2:310:38:032
178878995132,15cyclictest1-21systemd12:10:002
17888199508,5cyclictest131rcu_preempt09:30:413
17888199504,11cyclictest37-21ksoftirqd/309:56:173
17888199503,15cyclictest37-21ksoftirqd/310:09:553
17888199501,4cyclictest0-21swapper/311:45:023
178881995014,9cyclictest37-21ksoftirqd/312:13:363
178881995014,5cyclictest131rcu_preempt10:20:403
178881995010,9cyclictest37-21ksoftirqd/311:29:283
178878995043,5cyclictest181447-21kworker/2:210:00:482
178878995042,4cyclictest181447-21kworker/2:209:53:142
178878995034,13cyclictest3467602-21systemd-journal12:18:452
178878995033,13cyclictest275-21dbus-daemon07:40:012
17888199498,6cyclictest131rcu_preempt08:10:013
178881994941,6cyclictest361rcuc/312:23:133
178881994941,6cyclictest361rcuc/312:23:123
17888199493,9cyclictest220846-21ssh10:30:033
178881994912,5cyclictest131rcu_preempt12:30:503
178881994910,6cyclictest131rcu_preempt09:19:263
178878994941,5cyclictest257649-21kworker/2:212:03:302
178878994941,5cyclictest211010-21kworker/2:311:47:012
178878994941,5cyclictest211010-21kworker/2:311:11:242
178878994940,6cyclictest227087-21kworker/2:011:28:202
178878994940,6cyclictest184297-21kworker/2:010:15:002
178878994939,6cyclictest179281-21kworker/2:008:00:012
17887499498,26cyclictest271291-21kworker/u8:212:30:021
17888199487,5cyclictest131rcu_preempt11:36:553
17888199485,10cyclictest37-21ksoftirqd/311:33:243
17888199484,5cyclictest131rcu_preempt09:46:573
17888199481,8cyclictest0-21swapper/308:30:433
178881994814,5cyclictest131rcu_preempt08:45:013
178881994813,5cyclictest131rcu_preempt12:26:403
17888199481,18cyclictest0-21swapper/307:25:323
17888199481,11cyclictest37-21ksoftirqd/309:00:003
178878994841,4cyclictest181447-21kworker/2:208:55:012
178878994839,6cyclictest181447-21kworker/2:209:47:332
178878994836,9cyclictest30-21ksoftirqd/207:14:592
178874994839,6cyclictest23-21ksoftirqd/112:20:331
178874994839,6cyclictest23-21ksoftirqd/112:20:331
178874994834,10cyclictest3467602-21systemd-journal10:45:001
178866994818,25cyclictest16850irq/134-800000007:10:020
17888199475,5cyclictest131rcu_preempt11:05:213
178878994739,5cyclictest227087-21kworker/2:012:12:262
178878994739,5cyclictest227087-21kworker/2:011:05:262
178878994739,5cyclictest211010-21kworker/2:311:33:412
178878994739,5cyclictest181447-21kworker/2:209:59:392
178874994731,12cyclictest3467602-21systemd-journal10:15:091
178874994727,16cyclictest3467602-21systemd-journal10:30:001
178874994716,13cyclictest23-21ksoftirqd/111:13:291
17888199468,6cyclictest131rcu_preempt11:21:393
17888199464,9cyclictest37-21ksoftirqd/310:00:103
178881994630,13cyclictest1-21systemd07:20:003
178878994639,4cyclictest181447-21kworker/2:209:35:112
178878994638,5cyclictest270597-21cron12:25:002
178878994638,5cyclictest270597-21cron12:25:002
178878994638,5cyclictest227087-21kworker/2:012:31:192
178878994638,5cyclictest227087-21kworker/2:012:28:192
178878994638,5cyclictest227087-21kworker/2:011:04:332
178878994638,5cyclictest179281-21kworker/2:008:14:342
178878994637,5cyclictest227087-21kworker/2:011:52:162
178874994630,11cyclictest275-21dbus-daemon10:50:011
178874994629,12cyclictest275-21dbus-daemon10:25:011
178878994537,5cyclictest257649-21kworker/2:212:37:352
178878994536,6cyclictest184297-21kworker/2:009:27:212
178878994536,6cyclictest181447-21kworker/2:208:35:012
178874994532,9cyclictest275-21dbus-daemon08:36:521
178874994530,11cyclictest1-21systemd08:05:011
178874994527,12cyclictest23-21ksoftirqd/107:20:001
17888199447,11cyclictest37-21ksoftirqd/309:09:453
178881994426,12cyclictest3467602-21systemd-journal08:15:013
178878994437,5cyclictest211010-21kworker/2:311:41:432
178878994437,5cyclictest211010-21kworker/2:310:28:552
178878994437,4cyclictest181447-21kworker/2:210:07:332
178878994433,7cyclictest184563-21cron08:59:592
17887499443,5cyclictest131rcu_preempt11:17:481
178874994425,10cyclictest23-21ksoftirqd/112:15:571
17888199433,7cyclictest37-21ksoftirqd/307:55:013
17888199433,5cyclictest131rcu_preempt07:46:453
17888199431,11cyclictest0-21swapper/308:54:493
178878994337,4cyclictest211010-21kworker/2:310:40:392
178878994335,5cyclictest182682-21kworker/2:108:40:432
178878994332,8cyclictest3467602-21systemd-journal08:40:012
178878994327,12cyclictest3467602-21systemd-journal09:13:432
17887499434,6cyclictest131rcu_preempt10:57:311
178874994328,12cyclictest23-21ksoftirqd/111:30:031
178874994325,13cyclictest275-21dbus-daemon10:15:021
178874994322,9cyclictest23-21ksoftirqd/107:15:021
17888199427,9cyclictest37-21ksoftirqd/307:41:123
17888199425,7cyclictest0-21swapper/307:25:013
17887899429,10cyclictest247092-1kworker/u9:011:38:142
178878994228,9cyclictest185297-21cat09:10:032
178878994223,15cyclictest3467602-21systemd-journal09:25:002
178874994232,7cyclictest3467602-21systemd-journal08:07:441
178874994227,12cyclictest23-21ksoftirqd/108:35:001
178874994225,12cyclictest1-21systemd09:20:001
178874994224,13cyclictest23-21ksoftirqd/110:05:001
178874994219,19cyclictest23-21ksoftirqd/112:12:291
178874994218,19cyclictest23-21ksoftirqd/109:29:391
17888199412,6cyclictest0-21swapper/308:22:443
178878994124,12cyclictest3467602-21systemd-journal07:20:012
17887499414,5cyclictest131rcu_preempt09:55:021
17887499414,5cyclictest131rcu_preempt09:20:321
178874994129,8cyclictest23-21ksoftirqd/111:53:271
178874994126,10cyclictest275-21dbus-daemon10:55:011
178874994125,12cyclictest1-21systemd08:00:001
178874994124,13cyclictest3467602-21systemd-journal07:25:001
178874994119,5cyclictest221rcuc/107:45:011
178881994025,10cyclictest309-21in:imuxsock09:05:003
17888199401,11cyclictest37-21ksoftirqd/307:34:453
17888199401,11cyclictest37-21ksoftirqd/307:34:443
178878994033,5cyclictest211010-21kworker/2:311:24:142
178874994031,6cyclictest273345-21sh12:30:541
178874994028,9cyclictest3467602-21systemd-journal08:15:321
178874994027,9cyclictest236018-21cron11:05:001
178874994026,10cyclictest23-21ksoftirqd/111:23:211
178874994025,12cyclictest23-21ksoftirqd/109:49:021
178874994025,11cyclictest23-21ksoftirqd/109:33:061
178874994022,14cyclictest209175-21sh10:05:231
17887499401,8cyclictest0-21swapper/111:36:071
178874994017,5cyclictest23-21ksoftirqd/111:07:371
17887499401,6cyclictest0-21swapper/110:32:501
178874994015,17cyclictest23-21ksoftirqd/109:37:491
17887499401,3cyclictest131rcu_preempt09:55:081
17888199396,7cyclictest37-21ksoftirqd/308:20:033
178878993931,5cyclictest179281-21kworker/2:008:05:002
178878993925,10cyclictest309-21in:imuxsock07:45:012
17887499394,8cyclictest0-21swapper/108:25:011
17887499394,4cyclictest131rcu_preempt12:35:551
178874993926,10cyclictest23-21ksoftirqd/109:43:021
178874993923,12cyclictest23-21ksoftirqd/111:45:291
17887499391,7cyclictest0-21swapper/111:25:151
178874993913,18cyclictest23-21ksoftirqd/111:43:121
17888199384,8cyclictest37-21ksoftirqd/308:00:143
178878993824,11cyclictest30-21ksoftirqd/207:50:012
178878993821,12cyclictest275-21dbus-daemon08:25:012
178874993826,9cyclictest309-21in:imuxsock09:05:441
178874993824,11cyclictest181118-21sshd07:52:451
17888199371,9cyclictest37-21ksoftirqd/308:25:553
178878993720,12cyclictest275-21dbus-daemon08:50:012
17887499378,25cyclictest221rcuc/108:30:021
178874993728,5cyclictest183816-21munin-plugin-st08:45:001
178874993725,10cyclictest23-21ksoftirqd/107:50:021
17887899362,7cyclictest181907-21sendmail-mta08:09:292
178878993627,6cyclictest172134-21kworker/2:307:20:352
178874993626,7cyclictest23-21ksoftirqd/109:11:231
178874993625,8cyclictest23-21ksoftirqd/112:07:281
178874993620,12cyclictest23-21ksoftirqd/112:03:041
178878993525,6cyclictest182435-21sh08:20:002
178878993523,9cyclictest309-21in:imuxsock07:55:012
178874993516,6cyclictest23-21ksoftirqd/108:14:451
17887899341,11cyclictest179820-21sshd07:25:022
178874993427,4cyclictest184255-21latency_hist08:50:021
178874993424,7cyclictest23-21ksoftirqd/107:30:031
178874993424,7cyclictest23-21ksoftirqd/107:30:021
178874993423,8cyclictest1-21systemd09:00:011
178874993422,8cyclictest0-21swapper/107:25:031
17887499334,3cyclictest131rcu_preempt07:40:011
17887499332,5cyclictest131rcu_preempt09:02:521
178874993319,11cyclictest184413-21sed08:55:011
178881993223,7cyclictest37-21ksoftirqd/307:10:023
17887499303,6cyclictest131rcu_preempt07:10:011
178878992817,9cyclictest30-21ksoftirqd/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional