You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-13 - 23:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Mon Jul 13, 2026 12:45:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3132710991176,8cyclictest131rcu_preempt08:10:023
313270399795,67cyclictest3236997-21ssh12:35:031
313270399737,6cyclictest131rcu_preempt10:20:001
3132700997210,5cyclictest131rcu_preempt12:12:390
3132700997210,5cyclictest131rcu_preempt12:12:390
313270099698,6cyclictest131rcu_preempt11:10:110
3132700996826,10cyclictest3176333-21ssh10:28:140
313270099676,7cyclictest121ksoftirqd/011:50:020
313270099668,10cyclictest12-21ksoftirqd/011:55:020
3132700996537,7cyclictest3181891-21modprobe10:40:000
3132700996421,13cyclictest12-21ksoftirqd/009:20:010
3132700996353,8cyclictest0-21swapper/008:35:020
313270099611,10cyclictest12-21ksoftirqd/011:07:020
3132700996056,2cyclictest0-21swapper/007:50:020
3132700996034,8cyclictest0-21swapper/012:17:480
313270099603,11cyclictest12-21ksoftirqd/011:38:420
313270099594,13cyclictest12-21ksoftirqd/011:00:000
3132700995928,15cyclictest3199626-21ssh11:16:570
3132700995919,34cyclictest0-21swapper/009:48:430
3132700995914,23cyclictest12-21ksoftirqd/009:45:030
313270099585,46cyclictest0-21swapper/009:10:000
313270099585,46cyclictest0-21swapper/009:10:000
3132700995828,14cyclictest3169070-21ssh10:12:570
3132706995749,5cyclictest3136299-21kworker/2:012:04:202
3132706995749,5cyclictest3136299-21kworker/2:012:04:202
313270099574,9cyclictest12-21ksoftirqd/009:55:010
3132700995729,9cyclictest12-21ksoftirqd/009:11:150
3132700995713,26cyclictest3153312-21ssh09:40:020
3132700995710,24cyclictest131rcu_preempt10:05:030
313270699562,10cyclictest3467602-21systemd-journal08:49:592
313270099565,6cyclictest131rcu_preempt08:05:020
3132700995635,16cyclictest3467602-21systemd-journal10:55:010
3132700995634,17cyclictest16850irq/134-800000011:30:000
3132700995627,25cyclictest12-21ksoftirqd/009:27:230
3132700995625,15cyclictest3143919-21ssh09:20:100
3132700995546,6cyclictest12-21ksoftirqd/010:20:010
3132700995523,16cyclictest3192022-21ssh11:00:450
313270099551,7cyclictest121ksoftirqd/012:06:280
313270099549,28cyclictest131rcu_preempt10:20:120
3132700995435,13cyclictest1-21systemd11:25:010
3132700995427,12cyclictest3235097-21ssh12:31:000
3132700995427,12cyclictest3235097-21ssh12:31:000
3132700995426,13cyclictest3212006-21ssh11:42:480
313270099542,5cyclictest131rcu_preempt09:41:040
3132700995422,27cyclictest3184312-21cron10:45:000
3132700995420,5cyclictest16850irq/134-800000012:02:550
3132700995420,5cyclictest16850irq/134-800000012:02:540
313270699537,8cyclictest3136792-21cut08:30:002
3132706995312,11cyclictest131rcu_preempt11:46:532
3132700995329,20cyclictest3207646-21ssh11:33:440
3132700995324,24cyclictest3150557-21sh09:34:240
313270099531,10cyclictest12-21ksoftirqd/010:48:170
3132706995243,5cyclictest3137856-21kworker/2:309:23:122
3132706995226,23cyclictest30-21ksoftirqd/212:40:022
3132706995212,5cyclictest131rcu_preempt10:10:372
3132706995211,23cyclictest3189274-21ssh10:55:032
313270399523,7cyclictest131rcu_preempt08:05:001
3132703995235,13cyclictest1-21systemd10:40:001
313270099528,9cyclictest12-21ksoftirqd/012:40:020
313270099525,9cyclictest12-21ksoftirqd/008:10:000
313270099525,35cyclictest12-21ksoftirqd/008:56:200
313270099525,10cyclictest12-21ksoftirqd/012:00:010
3132700995217,7cyclictest12-21ksoftirqd/007:29:210
3132706995143,5cyclictest3136299-21kworker/2:010:43:192
3132706995142,5cyclictest3181958-21kworker/2:211:42:102
3132706995142,5cyclictest3119964-21kworker/2:107:20:012
3132706995112,9cyclictest131rcu_preempt12:31:182
3132706995112,9cyclictest131rcu_preempt12:31:182
3132700995132,14cyclictest3467602-21systemd-journal08:30:010
3132700995124,12cyclictest3231926-21ssh12:24:420
3132700995122,14cyclictest3233590-21ssh12:28:010
313270099511,5cyclictest0-21swapper/008:55:020
3132706995042,6cyclictest30-21ksoftirqd/207:40:022
3132706995042,5cyclictest3137856-21kworker/2:310:39:442
3132706995042,5cyclictest3137856-21kworker/2:309:38:132
3132706995042,5cyclictest3136299-21kworker/2:009:57:012
3132706995042,5cyclictest3136299-21kworker/2:009:57:012
3132706995034,12cyclictest3467602-21systemd-journal09:30:012
3132706995033,12cyclictest275-21dbus-daemon10:35:012
313270699502,12cyclictest3216173-21ssh11:51:272
3132703995012,22cyclictest3191591-21ssh11:00:011
3132703995010,6cyclictest131rcu_preempt12:00:001
313270099504,8cyclictest3134181-21modprobe07:40:000
313270099504,11cyclictest12-21ksoftirqd/010:00:010
313270099504,11cyclictest12-21ksoftirqd/010:00:010
313270099503,16cyclictest12-21ksoftirqd/008:20:000
3132700995030,16cyclictest3135220-21cat08:00:000
3132700995019,28cyclictest151rcuc/010:30:030
313271099497,39cyclictest3476162-21ntpd10:35:023
3132710994924,20cyclictest37-21ksoftirqd/312:00:003
3132706994941,5cyclictest3181958-21kworker/2:211:21:452
3132706994941,5cyclictest3181958-21kworker/2:211:17:422
3132706994941,5cyclictest3137856-21kworker/2:310:16:532
3132706994941,5cyclictest3137856-21kworker/2:309:18:492
3132706994941,5cyclictest3137856-21kworker/2:309:11:422
3132706994941,5cyclictest3136299-21kworker/2:011:58:502
3132706994941,5cyclictest3136299-21kworker/2:011:35:472
3132706994941,5cyclictest3136299-21kworker/2:010:48:382
3132706994941,5cyclictest3112785-21kworker/2:007:10:012
3132706994933,12cyclictest1-21systemd12:30:002
313270399495,3cyclictest131rcu_preempt07:55:011
313270399494,8cyclictest23-21ksoftirqd/110:55:021
313270399493,42cyclictest3148454-21sh09:30:011
313270099493,9cyclictest12-21ksoftirqd/008:40:010
3132710994836,8cyclictest3184324-21munin-plugin-st10:45:013
3132710994833,12cyclictest37-21ksoftirqd/309:35:333
3132710994832,12cyclictest275-21dbus-daemon12:30:013
3132710994831,12cyclictest3467602-21systemd-journal10:00:003
3132710994831,12cyclictest3467602-21systemd-journal10:00:003
3132710994813,8cyclictest0-21swapper/310:10:043
313270699485,7cyclictest30-21ksoftirqd/211:15:022
3132706994841,4cyclictest3181958-21kworker/2:212:08:032
3132706994840,5cyclictest3137856-21kworker/2:309:47:162
3132706994840,5cyclictest3137856-21kworker/2:309:35:012
3132706994840,5cyclictest3136299-21kworker/2:011:32:192
313270699482,6cyclictest3204172-21ssh11:26:162
313270699482,3cyclictest131rcu_preempt10:10:012
313270099486,7cyclictest12-21ksoftirqd/008:45:000
313270099485,10cyclictest12-21ksoftirqd/010:00:150
3132700994832,10cyclictest3467602-21systemd-journal07:25:000
3132710994740,5cyclictest37-21ksoftirqd/310:30:023
3132710994726,16cyclictest275-21dbus-daemon12:25:013
3132706994739,5cyclictest3181958-21kworker/2:212:12:252
3132706994739,5cyclictest3181958-21kworker/2:212:12:252
3132706994739,5cyclictest3136299-21kworker/2:010:50:592
3132706994738,6cyclictest3136299-21kworker/2:010:21:442
3132706994734,8cyclictest275-21dbus-daemon09:00:002
3132706994728,14cyclictest3467602-21systemd-journal07:15:002
313270099471,7cyclictest3134473-21unin-run07:45:010
313270099471,42cyclictest0-21swapper/007:15:010
3132710994636,5cyclictest3138863-21diskmemload11:35:003
3132710994632,10cyclictest1-21systemd12:15:013
3132710994632,10cyclictest1-21systemd12:15:013
3132710994610,6cyclictest131rcu_preempt08:30:013
3132706994638,5cyclictest3136299-21kworker/2:011:00:312
313270699463,4cyclictest131rcu_preempt11:05:372
313270699461,5cyclictest3158265-21ssh09:50:082
313270699461,5cyclictest131rcu_preempt12:18:472
3132703994633,9cyclictest3467602-21systemd-journal10:46:431
3132703994632,10cyclictest3134298-21date07:40:021
3132703994630,12cyclictest275-21dbus-daemon11:30:011
3132703994630,11cyclictest3467602-21systemd-journal08:29:591
3132703994628,12cyclictest275-21dbus-daemon08:40:011
313270399461,5cyclictest0-21swapper/111:37:341
3132706994537,5cyclictest3133963-21kworker/2:208:25:022
3132706994536,6cyclictest3135775-21kworker/2:308:15:162
3132706994532,9cyclictest3467602-21systemd-journal07:35:012
3132706994531,11cyclictest3138959-21sed09:10:022
3132706994531,11cyclictest3138959-21sed09:10:022
3132706994527,13cyclictest275-21dbus-daemon08:45:012
3132706994511,6cyclictest131rcu_preempt07:44:402
3132703994532,9cyclictest3161273-21sshd09:56:431
3132703994532,9cyclictest3161273-21sshd09:56:431
3132703994531,11cyclictest23-21ksoftirqd/111:52:141
3132703994530,11cyclictest23-21ksoftirqd/110:02:421
313270099459,10cyclictest12-21ksoftirqd/007:35:030
313270099457,9cyclictest12-21ksoftirqd/007:55:010
3132700994520,7cyclictest12-21ksoftirqd/008:15:010
3132700994512,8cyclictest0-21swapper/007:19:200
3132710994433,7cyclictest37-21ksoftirqd/311:21:553
3132710994432,7cyclictest3146041-21ssh09:25:023
3132710994427,12cyclictest3467602-21systemd-journal10:55:033
3132710994425,15cyclictest275-21dbus-daemon09:05:013
3132706994435,6cyclictest3136299-21kworker/2:010:00:252
313270399443,4cyclictest131rcu_preempt11:44:291
3132703994431,10cyclictest275-21dbus-daemon09:40:001
3132703994426,15cyclictest23-21ksoftirqd/109:15:291
3132700994429,12cyclictest3137805-21sshd08:48:420
313271099433,6cyclictest131rcu_preempt12:35:013
313271099433,6cyclictest131rcu_preempt12:35:013
3132710994331,6cyclictest309-21in:imuxsock08:05:003
3132710994328,12cyclictest37-21ksoftirqd/311:46:203
3132710994328,10cyclictest1-21systemd09:35:013
3132710994327,11cyclictest1-21systemd08:25:003
3132710994327,10cyclictest3467602-21systemd-journal10:40:003
3132710994327,10cyclictest309-21in:imuxsock10:45:033
3132710994325,14cyclictest3165155-21cron10:05:003
3132710994321,16cyclictest275-21dbus-daemon07:25:013
313271099431,5cyclictest131rcu_preempt10:10:013
3132710994313,7cyclictest131rcu_preempt08:35:023
3132706994336,4cyclictest3181958-21kworker/2:212:21:142
3132706994335,5cyclictest3137856-21kworker/2:309:40:252
3132706994335,5cyclictest3137856-21kworker/2:309:40:252
3132706994310,4cyclictest131rcu_preempt10:25:302
313270399439,22cyclictest23-21ksoftirqd/108:10:021
313270399434,4cyclictest131rcu_preempt10:25:011
313270399433,10cyclictest0-21swapper/111:03:431
3132703994330,10cyclictest3182930-21ssh10:41:521
3132703994327,11cyclictest23-21ksoftirqd/109:46:401
313270399432,22cyclictest0-21swapper/108:53:511
313270399431,9cyclictest23-21ksoftirqd/107:35:011
313270099435,7cyclictest12-21ksoftirqd/008:25:020
313271099429,5cyclictest131rcu_preempt11:30:033
313271099425,6cyclictest0-21swapper/312:38:473
313271099425,3cyclictest131rcu_preempt11:18:463
313271099424,3cyclictest131rcu_preempt11:00:563
3132710994230,8cyclictest275-21dbus-daemon07:45:013
3132710994228,11cyclictest275-21dbus-daemon11:10:013
3132710994228,10cyclictest37-21ksoftirqd/307:50:023
313271099422,5cyclictest131rcu_preempt12:10:003
3132710994219,5cyclictest37-21ksoftirqd/311:43:413
313270699423,7cyclictest3467602-21systemd-journal07:50:012
3132706994234,5cyclictest3134739-21kworker/2:108:05:522
313270399426,4cyclictest131rcu_preempt11:50:011
3132703994229,9cyclictest3167607-21systemd-run10:10:001
3132703994229,10cyclictest3467602-21systemd-journal10:15:001
3132703994228,9cyclictest23-21ksoftirqd/111:08:041
3132703994228,11cyclictest23-21ksoftirqd/112:25:011
3132703994226,12cyclictest275-21dbus-daemon12:15:011
3132703994226,12cyclictest275-21dbus-daemon12:15:011
313270399421,6cyclictest0-21swapper/111:11:051
3132710994127,10cyclictest37-21ksoftirqd/309:10:073
3132710994121,12cyclictest309-21in:imuxsock09:05:033
3132710994121,12cyclictest309-21in:imuxsock09:05:023
3132703994129,9cyclictest23-21ksoftirqd/110:27:501
3132703994124,13cyclictest3467602-21systemd-journal09:21:331
313270399412,3cyclictest131rcu_preempt12:05:021
313270399412,3cyclictest131rcu_preempt12:05:021
313270399411,3cyclictest131rcu_preempt09:55:021
313270399411,11cyclictest3179091-21ssh10:34:081
313270099413,6cyclictest12-21ksoftirqd/009:05:030
313271099409,6cyclictest131rcu_preempt10:55:003
313271099405,5cyclictest131rcu_preempt07:20:033
313271099403,3cyclictest131rcu_preempt12:03:193
313271099403,3cyclictest131rcu_preempt12:03:193
3132710994027,9cyclictest37-21ksoftirqd/312:19:403
3132710994025,10cyclictest309-21in:imuxsock09:45:033
3132710994024,12cyclictest1-21systemd11:40:003
313270699404,5cyclictest131rcu_preempt07:58:402
313270699401,6cyclictest0-21swapper/207:54:562
313270399404,4cyclictest131rcu_preempt11:15:141
313270399404,3cyclictest131rcu_preempt12:28:401
313270399403,4cyclictest131rcu_preempt08:20:021
3132703994027,9cyclictest23-21ksoftirqd/112:35:021
3132703994027,9cyclictest23-21ksoftirqd/112:35:021
3132703994026,11cyclictest3154605-21sshd09:42:421
3132703994026,11cyclictest3154605-21sshd09:42:421
3132703994023,13cyclictest275-21dbus-daemon07:20:001
313270399401,4cyclictest131rcu_preempt12:20:001
3132710993929,7cyclictest37-21ksoftirqd/309:25:323
3132710993928,9cyclictest37-21ksoftirqd/311:15:023
3132710993927,8cyclictest309-21in:imuxsock08:11:403
3132710993924,11cyclictest275-21dbus-daemon07:40:013
3132710993923,11cyclictest3467602-21systemd-journal09:00:003
3132710993923,11cyclictest0-21swapper/308:40:013
3132710993922,12cyclictest3467602-21systemd-journal08:00:003
313271099391,5cyclictest3143686-21ssh09:20:023
313271099391,34cyclictest37-21ksoftirqd/308:50:013
313270699396,3cyclictest131rcu_preempt08:05:002
3132703993931,5cyclictest23-21ksoftirqd/109:12:271
3132703993927,8cyclictest3138841-21sshd09:08:421
3132703993927,8cyclictest3138841-21sshd09:08:421
313270399392,28cyclictest3132704-21latency_hist07:10:021
3132703993921,13cyclictest3138324-21modprobe09:00:001
3132703993916,4cyclictest23-21ksoftirqd/108:15:001
313270399391,5cyclictest131rcu_preempt08:50:001
3132703993911,6cyclictest131rcu_preempt11:32:441
313271099389,6cyclictest131rcu_preempt07:15:023
313271099384,3cyclictest131rcu_preempt11:50:113
3132710993826,9cyclictest37-21ksoftirqd/309:40:313
3132710993826,9cyclictest37-21ksoftirqd/309:40:303
3132710993815,19cyclictest37-21ksoftirqd/310:20:013
3132706993830,5cyclictest3135775-21kworker/2:308:15:012
313270399384,3cyclictest131rcu_preempt08:45:011
3132703993827,8cyclictest3137026-21sshd08:34:401
3132703993826,9cyclictest3136660-21sed08:25:021
313270399382,4cyclictest131rcu_preempt09:33:271
3132703993820,12cyclictest275-21dbus-daemon07:50:011
313270399381,3cyclictest131rcu_preempt11:21:581
3132703993811,6cyclictest131rcu_preempt12:09:061
313271099373,6cyclictest131rcu_preempt07:30:003
3132710993727,8cyclictest37-21ksoftirqd/310:25:023
3132710993727,7cyclictest37-21ksoftirqd/309:50:023
3132710993727,5cyclictest37-21ksoftirqd/308:45:003
3132706993727,8cyclictest30-21ksoftirqd/207:24:192
3132706993726,6cyclictest309-21in:imuxsock08:34:592
3132706993724,10cyclictest30-21ksoftirqd/208:40:012
3132703993725,9cyclictest3133656-21sshd07:28:211
3132703993725,9cyclictest275-21dbus-daemon07:45:011
3132710993625,8cyclictest3133942-21systemctl07:35:013
313270399362,3cyclictest131rcu_preempt08:00:001
3132703993622,10cyclictest23-21ksoftirqd/109:05:021
3132706993529,3cyclictest3132903-21kworker/2:307:27:562
3132706993528,4cyclictest3138075-21latency_hist08:55:022
313270399354,4cyclictest131rcu_preempt07:12:321
313270399353,3cyclictest131rcu_preempt07:24:511
313271099344,5cyclictest131rcu_preempt07:55:003
3132710993419,11cyclictest275-21dbus-daemon08:54:593
3132710993418,6cyclictest131rcu_preempt08:20:003
3132706993426,5cyclictest3138606-21latency_hist09:05:012
3132710993322,5cyclictest37-21ksoftirqd/307:10:033
3132700993024,4cyclictest3132632-21latency_hist07:10:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional