You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-08 - 03:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Mon Jun 08, 2026 00:45:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
64804998932,13cyclictest72719-21ssh21:13:530
64804998736,47cyclictest16950irq/224-800000020:20:010
64804997625,47cyclictest12-21ksoftirqd/021:40:340
64804997625,47cyclictest12-21ksoftirqd/021:40:340
64804997527,13cyclictest152672-21ssh00:14:370
64804997516,41cyclictest146180-40mandb00:00:030
64804997516,41cyclictest146180-40mandb00:00:030
6480499744,10cyclictest131rcu_preempt22:50:010
6480499721,42cyclictest91320-21ssh21:55:300
64804997130,14cyclictest275-21dbus-daemon23:59:590
64804997029,9cyclictest12-21ksoftirqd/019:40:020
64804997025,20cyclictest16850irq/134-800000022:30:020
64804996829,11cyclictest12-21ksoftirqd/023:46:460
6480499673,4cyclictest131rcu_preempt00:40:000
64812996610,15cyclictest99531-21ssh22:14:211
64804996521,9cyclictest74287-21ssh21:17:150
64812996421,8cyclictest23-21ksoftirqd/121:30:011
6480499641,9cyclictest79409-21ssh21:28:580
64821996331,13cyclictest37-21ksoftirqd/300:00:023
6480499635,41cyclictest70987-21diskmemload22:35:020
64804996329,13cyclictest103286-21ssh22:22:300
64804996314,27cyclictest12-21ksoftirqd/000:31:160
6480499631,40cyclictest120898-21ssh23:02:250
6481299621,18cyclictest86484-21munin-run21:45:011
6481299621,18cyclictest86484-21munin-run21:45:001
64812996119,9cyclictest23-21ksoftirqd/100:25:351
6481299607,15cyclictest311-21rs:main1
64804996029,27cyclictest130745-21ssh23:25:000
64812995913,8cyclictest23-21ksoftirqd/119:30:021
64812995912,8cyclictest23-21ksoftirqd/122:05:021
64816995848,7cyclictest58225-21kworker/2:200:08:142
6481299588,8cyclictest23-21ksoftirqd/100:00:001
64816995749,5cyclictest58225-21kworker/2:221:35:472
64816995748,6cyclictest58225-21kworker/2:221:29:142
6481299578,8cyclictest23-21ksoftirqd/119:35:021
6481299577,7cyclictest23-21ksoftirqd/123:31:421
6481299577,14cyclictest142106-21rm23:50:201
64812995715,9cyclictest23-21ksoftirqd/121:31:161
64804995718,36cyclictest12-21ksoftirqd/022:00:310
6481299569,8cyclictest23-21ksoftirqd/122:30:021
6481299568,13cyclictest115925-21ssh22:50:541
6481299565,11cyclictest23-21ksoftirqd/123:20:021
64812995636,15cyclictest3467602-21systemd-journal23:40:001
64812995610,8cyclictest23-21ksoftirqd/100:12:271
6480499565,48cyclictest16850irq/134-800000023:10:020
6480499564,31cyclictest131rcu_preempt19:44:370
64816995547,5cyclictest84343-21kworker/2:122:15:192
64816995547,5cyclictest58225-21kworker/2:222:55:262
64816995513,11cyclictest30-21ksoftirqd/221:45:012
64816995513,11cyclictest30-21ksoftirqd/221:45:002
6481299559,10cyclictest23-21ksoftirqd/119:45:021
6481299558,10cyclictest23-21ksoftirqd/123:45:021
64812995514,8cyclictest23-21ksoftirqd/121:23:451
64812995510,12cyclictest90225-21sh21:53:161
64804995545,6cyclictest16950irq/224-800000020:00:000
64816995447,4cyclictest58225-21kworker/2:223:32:202
64816995446,5cyclictest84343-21kworker/2:122:30:162
64816995445,6cyclictest58225-21kworker/2:200:25:002
64816995445,5cyclictest84343-21kworker/2:121:58:472
64816995445,5cyclictest58225-21kworker/2:200:29:352
6481299544,10cyclictest23-21ksoftirqd/120:50:021
64812995416,8cyclictest23-21ksoftirqd/121:20:021
64812995412,8cyclictest23-21ksoftirqd/123:11:261
64821995343,6cyclictest69419-21sendmail20:40:003
64816995345,5cyclictest58225-21kworker/2:223:15:292
64816995345,5cyclictest58225-21kworker/2:200:18:542
64816995345,4cyclictest58225-21kworker/2:221:11:382
64816995344,6cyclictest58225-21kworker/2:222:45:422
64816995344,5cyclictest84343-21kworker/2:122:12:152
64816995344,5cyclictest58225-21kworker/2:223:35:382
64816995343,6cyclictest58225-21kworker/2:223:24:022
6481299538,7cyclictest111998-21ssh22:42:101
6481299537,9cyclictest120371-21ssh23:01:151
6481299537,8cyclictest23-21ksoftirqd/122:40:001
6481299537,12cyclictest147513-21sh00:02:481
6481299537,12cyclictest147513-21sh00:02:481
6481299535,13cyclictest23-21ksoftirqd/100:15:251
6481299533,14cyclictest82437-21sh21:35:351
64812995323,8cyclictest131rcu_preempt21:55:511
64812995314,10cyclictest23-21ksoftirqd/100:05:381
6481299531,15cyclictest23-21ksoftirqd/122:18:251
64812995310,10cyclictest0-21swapper/119:55:031
6480499536,42cyclictest70986-21sh21:10:000
6480499533,29cyclictest131rcu_preempt20:13:370
64804995327,11cyclictest76737-21ssh21:22:450
64804995324,13cyclictest82243-21ssh21:35:090
64816995244,5cyclictest58225-21kworker/2:223:40:502
64816995244,5cyclictest58225-21kworker/2:222:07:132
64816995244,5cyclictest58225-21kworker/2:219:20:012
64816995244,5cyclictest58225-21kworker/2:200:04:522
64816995244,5cyclictest58225-21kworker/2:200:04:512
64816995243,5cyclictest58225-21kworker/2:219:35:022
64816995242,7cyclictest58225-21kworker/2:223:03:442
64816995242,6cyclictest126368-21kworker/2:100:11:572
64816995236,12cyclictest160445-21sshd00:31:512
6481299528,9cyclictest23-21ksoftirqd/100:30:331
64812995232,13cyclictest163882-21cat00:40:001
64804995248,2cyclictest0-21swapper/019:15:010
6480499523,3cyclictest131rcu_preempt21:52:350
6480499523,3cyclictest131rcu_preempt21:32:420
64804995219,30cyclictest12-21ksoftirqd/022:39:060
64816995143,5cyclictest58225-21kworker/2:223:10:002
64816995143,5cyclictest58225-21kworker/2:222:03:392
64816995143,5cyclictest58225-21kworker/2:221:33:492
64816995142,5cyclictest84343-21kworker/2:121:54:362
64816995141,7cyclictest58225-21kworker/2:221:16:292
6481299512,14cyclictest87749-21sh21:47:371
64812995112,10cyclictest23-21ksoftirqd/122:09:021
6481299511,17cyclictest156697-21sh00:23:331
6480499512,5cyclictest66046-21modprobe19:35:000
64804995121,14cyclictest116141-21ssh22:51:340
64816995042,5cyclictest58225-21kworker/2:223:10:122
64816995042,5cyclictest58225-21kworker/2:222:27:422
64816995042,5cyclictest58225-21kworker/2:221:46:292
64816995042,5cyclictest58225-21kworker/2:219:15:012
64816995041,6cyclictest58225-21kworker/2:221:24:332
64816995034,11cyclictest1-21systemd22:55:012
6481299505,9cyclictest72133-21rm21:12:181
6481299503,9cyclictest23-21ksoftirqd/123:27:071
64804995020,27cyclictest12-21ksoftirqd/023:15:000
64816994941,5cyclictest58225-21kworker/2:223:47:092
64816994941,5cyclictest58225-21kworker/2:222:21:092
64816994941,5cyclictest58225-21kworker/2:200:00:002
64816994940,6cyclictest84343-21kworker/2:122:36:302
64816994931,12cyclictest30-21ksoftirqd/200:40:002
6481299497,7cyclictest23-21ksoftirqd/121:10:011
64812994940,6cyclictest221rcuc/122:22:121
6481299493,9cyclictest23-21ksoftirqd/122:46:451
64812994914,7cyclictest23-21ksoftirqd/122:58:571
64812994912,9cyclictest23-21ksoftirqd/123:45:371
6480499492,5cyclictest70264-21latency_hist20:55:020
64804994923,11cyclictest117872-21ssh22:55:210
6480499491,3cyclictest131rcu_preempt23:19:180
64821994832,12cyclictest3467602-21systemd-journal00:35:003
6482199482,10cyclictest163884-21run-parts00:40:013
64816994840,5cyclictest84343-21kworker/2:122:40:302
64816994840,5cyclictest58225-21kworker/2:223:54:392
64816994840,5cyclictest58225-21kworker/2:220:50:012
64816994840,5cyclictest58225-21kworker/2:219:55:012
64816994839,6cyclictest58225-21kworker/2:219:50:012
64816994839,5cyclictest58225-21kworker/2:223:25:252
64816994832,11cyclictest3467602-21systemd-journal20:05:012
64816994832,11cyclictest3467602-21systemd-journal20:05:012
6481299485,7cyclictest23-21ksoftirqd/123:08:081
6481299484,8cyclictest23-21ksoftirqd/120:25:041
6480499483,4cyclictest131rcu_preempt00:16:260
64804994828,16cyclictest111981-21ssh22:42:090
6480499481,3cyclictest131rcu_preempt20:10:020
64821994736,7cyclictest102034-21grep22:20:003
6481299474,12cyclictest23-21ksoftirqd/120:20:021
64812994733,10cyclictest3467602-21systemd-journal20:31:381
64812994710,7cyclictest0-21swapper/120:05:011
64812994710,7cyclictest0-21swapper/120:05:011
64804994724,8cyclictest99759-21ssh22:14:540
64804994721,11cyclictest88220-21ssh21:48:500
64804994711,33cyclictest137341-21latency_hist23:40:010
6482199462,3cyclictest131rcu_preempt23:50:303
64821994616,5cyclictest346771850lldpd22:54:453
64816994638,5cyclictest58225-21kworker/2:220:59:262
64816994637,6cyclictest58225-21kworker/2:220:35:052
6481299466,9cyclictest23-21ksoftirqd/123:23:381
64812994612,10cyclictest23-21ksoftirqd/121:00:001
64804994630,12cyclictest1-21systemd23:30:010
64804994629,13cyclictest159257-21ssh00:29:300
64804994628,14cyclictest148915-21ssh00:05:460
64804994628,14cyclictest134206-21sshd23:32:380
6482199456,5cyclictest131rcu_preempt21:00:003
64821994531,11cyclictest37-21ksoftirqd/323:43:393
64821994531,11cyclictest1-21systemd20:45:003
6482199452,3cyclictest131rcu_preempt21:54:293
6482199451,6cyclictest131rcu_preempt20:25:003
64816994538,5cyclictest58225-21kworker/2:220:37:112
64816994537,5cyclictest58225-21kworker/2:220:25:012
64816994537,5cyclictest58225-21kworker/2:219:55:572
6481299451,5cyclictest0-21swapper/119:15:001
64804994537,5cyclictest16850irq/134-800000022:20:000
64804994529,13cyclictest142474-21ssh23:51:110
64804994528,13cyclictest96959-21ssh22:08:250
64804994528,13cyclictest155989-21sshd00:21:470
6480499452,13cyclictest139155-21ssh23:43:520
64821994432,8cyclictest3467602-21systemd-journal21:18:323
6482199442,5cyclictest131rcu_preempt21:40:223
6482199442,5cyclictest131rcu_preempt21:40:223
64821994418,5cyclictest70987-21diskmemload22:28:223
6482199441,3cyclictest131rcu_preempt21:26:133
64816994436,5cyclictest58225-21kworker/2:221:03:472
64816994435,5cyclictest58225-21kworker/2:219:40:012
6481299446,9cyclictest23-21ksoftirqd/120:53:381
64812994435,7cyclictest23-21ksoftirqd/119:20:011
6482199432,3cyclictest131rcu_preempt20:05:013
6482199432,3cyclictest131rcu_preempt20:05:013
6481699435,5cyclictest131rcu_preempt21:10:012
64816994336,4cyclictest58225-21kworker/2:220:18:212
6481299435,12cyclictest23-21ksoftirqd/119:55:021
6481299432,10cyclictest23-21ksoftirqd/120:45:011
64804994324,13cyclictest275-21dbus-daemon20:30:000
6482199427,6cyclictest131rcu_preempt22:00:023
64821994232,6cyclictest137342-21sendmail23:40:013
64821994229,10cyclictest37-21ksoftirqd/322:00:363
6482199422,7cyclictest131rcu_preempt22:25:013
6482199422,3cyclictest131rcu_preempt21:36:093
64821994213,19cyclictest37-21ksoftirqd/322:33:203
6481699424,5cyclictest131rcu_preempt20:30:012
6481699422,6cyclictest131rcu_preempt20:15:012
6481299427,8cyclictest0-21swapper/119:25:011
64812994234,5cyclictest23-21ksoftirqd/121:05:021
64812994226,11cyclictest275-21dbus-daemon20:15:011
64804994235,5cyclictest0-21swapper/020:25:000
6480499422,35cyclictest0-21swapper/019:28:370
6482199416,4cyclictest131rcu_preempt00:20:013
6482199414,5cyclictest131rcu_preempt20:35:013
64821994129,9cyclictest1-21systemd21:35:003
64821994126,12cyclictest37-21ksoftirqd/323:17:093
64821994125,11cyclictest37-21ksoftirqd/321:48:043
6482199411,4cyclictest131rcu_preempt22:15:003
6482199411,3cyclictest131rcu_preempt00:03:033
6482199411,3cyclictest131rcu_preempt00:03:033
64816994133,5cyclictest58225-21kworker/2:220:55:012
64816994130,7cyclictest67845-21sshd20:07:382
64816994123,4cyclictest30-21ksoftirqd/219:30:012
6480499414,5cyclictest69421-21sh20:40:000
6480499412,32cyclictest70442-21modprobe21:00:000
64804994123,14cyclictest3467602-21systemd-journal20:35:010
64804994110,27cyclictest12-21ksoftirqd/020:01:140
64804994110,27cyclictest12-21ksoftirqd/020:01:140
6482199404,3cyclictest131rcu_preempt23:10:023
64821994031,5cyclictest309-21in:imuxsock19:55:033
64821994028,9cyclictest97748-21date22:10:013
64821994026,11cyclictest37-21ksoftirqd/300:10:303
64821994026,10cyclictest115464-21grep22:50:023
6482199402,3cyclictest131rcu_preempt00:06:293
64821994023,14cyclictest70290-21sed20:55:023
6482199401,7cyclictest70987-21diskmemload23:30:013
6482199401,5cyclictest131rcu_preempt23:31:093
6482199401,3cyclictest131rcu_preempt19:35:013
64816994032,5cyclictest58225-21kworker/2:219:42:542
64816994032,5cyclictest58225-21kworker/2:219:21:482
6481299404,6cyclictest23-21ksoftirqd/120:10:011
6480499401,13cyclictest65526-21sshd19:23:370
64821993929,7cyclictest37-21ksoftirqd/323:00:323
64821993928,7cyclictest111754-21sshd22:41:393
64821993928,6cyclictest309-21in:imuxsock19:55:013
64821993927,9cyclictest37-21ksoftirqd/319:30:013
64821993927,10cyclictest37-21ksoftirqd/322:35:393
64821993925,10cyclictest37-21ksoftirqd/321:10:233
64821993925,10cyclictest141834-21systemd-run23:50:003
6482199392,3cyclictest131rcu_preempt20:20:023
64816993918,6cyclictest30-21ksoftirqd/220:45:002
6481299394,11cyclictest23-21ksoftirqd/119:40:031
6481299392,10cyclictest23-21ksoftirqd/120:26:381
6482199386,3cyclictest131rcu_preempt22:56:333
6482199385,18cyclictest0-21swapper/300:26:113
64821993828,6cyclictest65272-21munin-plugin-st19:20:003
64821993824,11cyclictest37-21ksoftirqd/320:14:373
64821993824,10cyclictest37-21ksoftirqd/321:22:493
6482199382,3cyclictest131rcu_preempt19:25:023
6482199381,5cyclictest131rcu_preempt19:50:023
6482199381,3cyclictest131rcu_preempt23:22:433
6482199381,21cyclictest0-21swapper/300:20:133
6480499381,33cyclictest0-21swapper/019:53:370
6482199375,3cyclictest131rcu_preempt23:14:093
64821993725,10cyclictest37-21ksoftirqd/321:10:023
64821993724,10cyclictest275-21dbus-daemon19:15:003
64821993721,11cyclictest3467602-21systemd-journal19:44:023
64821993719,13cyclictest275-21dbus-daemon20:10:003
6481299375,8cyclictest23-21ksoftirqd/120:40:011
64804993728,5cyclictest309-21in:imuxsock20:41:580
6480499371,7cyclictest64889-21latency_hist19:10:020
6480499371,7cyclictest64889-21latency_hist19:10:020
6480499371,5cyclictest66838-21munin-run19:50:000
64821993629,4cyclictest68910-21latency_hist20:30:003
64804993625,8cyclictest65386-21sed19:20:020
6480499361,31cyclictest0-21swapper/020:45:020
6482199354,3cyclictest131rcu_preempt19:10:033
6482199354,3cyclictest131rcu_preempt19:10:023
64821993521,11cyclictest37-21ksoftirqd/321:04:443
64812993525,7cyclictest23-21ksoftirqd/119:10:021
64812993525,7cyclictest23-21ksoftirqd/119:10:021
64804993527,5cyclictest19350irq/34-mmc121:02:270
6482199344,3cyclictest131rcu_preempt19:39:563
64821993423,8cyclictest69922-21sshd20:47:383
6481299323,8cyclictest23-21ksoftirqd/119:47:371
64816992619,5cyclictest64744-21latency_hist19:10:022
64816992619,5cyclictest64744-21latency_hist19:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional