You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-19 - 19:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Tue May 19, 2026 12:45:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22008949910699,3cyclictest0-21swapper/010:00:030
22009069910394,6cyclictest37-21ksoftirqd/308:30:023
220089499911,83cyclictest0-21swapper/008:20:020
2200894999030,39cyclictest2271808-21ssh11:23:420
220089499831,4cyclictest2254494-21sshd10:47:540
220089499821,54cyclictest0-21swapper/010:15:010
2200894998126,51cyclictest2226390-21ssh09:49:450
220089499791,9cyclictest284-21systemd-logind11:20:010
2200894997918,55cyclictest16850irq/134-800000010:30:350
2200897997824,14cyclictest2233076-21sh10:03:321
220089499783,40cyclictest2219384-21grep09:35:020
2200894997830,44cyclictest2240371-21rm10:18:340
2200894997829,45cyclictest2245146-21ssh10:28:340
2200894997821,48cyclictest16850irq/134-800000012:39:270
220089499771,13cyclictest2267732-21cut11:15:030
2200894997627,45cyclictest2262076-21ssh11:03:400
2200894997618,54cyclictest2249157-21ssh10:36:390
220089499761,3cyclictest150-21pr/ttyS212:25:050
2200894997529,28cyclictest2283975-21ssh11:49:020
2200894997521,45cyclictest16850irq/134-800000012:33:160
220089499751,2cyclictest0-21swapper/009:25:020
2200906997329,38cyclictest275-21dbus-daemon10:15:013
2200901997313,19cyclictest2269924-21modprobe11:20:002
220089499732,66cyclictest2203424-21cron08:00:000
2200894997321,48cyclictest2274301-21ssh11:28:560
220089499721,2cyclictest0-21swapper/011:35:310
2200897997120,19cyclictest0-21swapper/109:40:031
2200894997129,25cyclictest2221391-21ssh09:39:260
2200894997124,43cyclictest0-21swapper/012:05:070
220089499712,43cyclictest2296516-21munin-run12:15:010
2200894997123,43cyclictest16850irq/134-800000011:06:090
2200894997118,36cyclictest2252495-21ssh10:43:420
2200894997117,51cyclictest2228345-21ssh09:53:500
2200894997021,30cyclictest16850irq/134-800000009:13:410
2200906996928,37cyclictest275-21dbus-daemon07:15:013
2200897996917,19cyclictest2265052-21sshd11:09:541
2200894996921,43cyclictest16850irq/134-800000010:20:120
220089499691,59cyclictest0-21swapper/008:30:020
2200906996827,36cyclictest309-21in:imuxsock09:00:003
2200897996822,15cyclictest2272392-21latency_hist11:25:021
2200894996820,44cyclictest16850irq/134-800000012:00:380
2200894996820,28cyclictest16850irq/134-800000011:56:320
2200906996723,8cyclictest2203405-21sshd07:56:543
220089499672,4cyclictest2205812-21sshd08:41:540
220089499671,62cyclictest0-21swapper/012:25:260
2200894996715,27cyclictest16850irq/134-800000009:43:020
2200894996715,27cyclictest16850irq/134-800000009:43:010
2200906996629,34cyclictest120-21kswapd010:05:083
2200906996623,6cyclictest2229195-21sh09:55:083
220089499664,43cyclictest19350irq/34-mmc112:15:030
2200894996623,39cyclictest2255952-21ssh10:50:380
2200906996526,9cyclictest2201342-21cron07:20:013
220089499658,52cyclictest16850irq/134-800000009:18:120
220089499652,7cyclictest2216843-21munin-run09:30:010
220089499652,15cyclictest2235993-21ssh10:09:460
2200906996422,10cyclictest37-21ksoftirqd/311:15:013
2200906996422,10cyclictest1-21systemd08:50:003
2200897996421,13cyclictest2247418-21ssh10:33:101
220089499642,57cyclictest2206108-21munin-run08:50:010
2200894996424,25cyclictest2258927-21ssh10:56:590
2200897996321,13cyclictest2248551-21rm10:35:141
2200897996315,17cyclictest2209206-21ssh09:14:101
2200894996311,33cyclictest16850irq/134-800000011:51:000
2200906996226,10cyclictest37-21ksoftirqd/309:18:483
2200906996222,11cyclictest37-21ksoftirqd/310:59:213
2200906996216,14cyclictest275-21dbus-daemon09:15:013
2200906996213,6cyclictest0-21swapper/311:20:003
2200894996221,37cyclictest2276340-21ssh11:33:040
2200906996119,9cyclictest37-21ksoftirqd/311:05:033
220089499612,12cyclictest48-21kcompactd011:40:050
220089499611,4cyclictest2233656-21sshd10:04:530
220089499611,14cyclictest2205342-21latency_hist08:35:020
220089499611,14cyclictest2205342-21latency_hist08:35:010
2200906996023,9cyclictest37-21ksoftirqd/311:10:023
220089499602,6cyclictest2206608-21munin-run09:00:000
2200906995925,8cyclictest2206323-21sshd08:53:493
2200906995923,9cyclictest37-21ksoftirqd/309:26:133
2200906995917,16cyclictest275-21dbus-daemon11:30:003
220089499591,5cyclictest2202195-21cat07:35:010
2200897995813,14cyclictest2239023-21ssh10:15:381
2200906995724,10cyclictest37-21ksoftirqd/312:01:213
2200906995719,9cyclictest3467602-21systemd-journal07:25:543
2200906995719,9cyclictest3467602-21systemd-journal07:25:533
2200906995716,10cyclictest3467602-21systemd-journal08:40:013
2200897995712,14cyclictest2216696-21ssh09:29:471
220089499572,50cyclictest2205603-21/usr/sbin/munin08:40:010
220089499571,5cyclictest2203729-21munin-run08:05:010
220089499571,4cyclictest2207152-21/usr/sbin/munin09:10:010
220089499571,4cyclictest2202358-21sshd07:35:540
220089499571,4cyclictest2202358-21sshd07:35:540
220089499571,4cyclictest0-21swapper/009:05:010
2200906995625,5cyclictest2302119-21sh12:26:273
220090699561,29cyclictest37-21ksoftirqd/307:50:003
2200897995640,13cyclictest2291872-21ssh12:05:041
2200906995526,8cyclictest37-21ksoftirqd/308:15:013
2200906995523,9cyclictest37-21ksoftirqd/312:35:023
2200906995520,8cyclictest37-21ksoftirqd/310:03:183
2200906995510,9cyclictest0-21swapper/307:25:003
2200901995537,13cyclictest3467602-21systemd-journal12:35:002
220090199551,37cyclictest2241049-21kworker/u8:010:30:012
2200897995521,8cyclictest2273684-21ssh11:27:291
2200897995515,12cyclictest0-21swapper/112:00:411
220089499551,7cyclictest2204737-21sshd08:23:540
2200906995424,4cyclictest2213691-21sh09:23:163
2200906995421,8cyclictest37-21ksoftirqd/312:16:073
2200906995411,8cyclictest0-21swapper/309:40:013
2200897995413,12cyclictest2207132-21diskmemload11:37:141
2200897995412,10cyclictest0-21swapper/110:26:381
220089499542,6cyclictest2204198-21sshd08:13:550
220089499542,5cyclictest2201365-21munin-run07:20:010
220089499541,5cyclictest2201061-21sshd07:10:540
2200906995323,8cyclictest37-21ksoftirqd/311:35:583
2200897995338,12cyclictest2241670-21ssh10:21:041
2200897995338,12cyclictest2214291-21ssh09:24:381
2200897995336,13cyclictest2297094-21ssh12:15:581
2200897995327,22cyclictest2210994-21ssh09:17:411
2200897995314,13cyclictest2266462-21ssh11:12:431
2200894995340,4cyclictest150-21pr/ttyS207:25:020
220089499531,4cyclictest2203167-21sshd07:54:550
220089499531,4cyclictest2201868-21sshd07:28:540
220089499531,4cyclictest2201868-21sshd07:28:530
2200901995244,5cyclictest2282120-21kworker/2:211:57:422
2200901995244,5cyclictest2282120-21kworker/2:211:49:302
2200901995233,13cyclictest2203425-21cron08:00:012
2200897995237,12cyclictest2294365-21ssh12:10:171
2200897995237,12cyclictest2279895-21ssh11:40:041
2200897995229,18cyclictest2230998-21ssh09:59:251
2200897995224,23cyclictest2256560-21sh10:52:041
2200897995210,13cyclictest2207132-21diskmemload12:32:241
2200894995220,28cyclictest2203965-21latency_hist08:10:030
220089499521,47cyclictest2206328-21sshd08:53:550
2200906995138,10cyclictest3467602-21systemd-journal10:40:003
2200906995123,7cyclictest37-21ksoftirqd/310:27:543
2200906995123,10cyclictest37-21ksoftirqd/309:48:273
2200906995120,10cyclictest37-21ksoftirqd/311:45:233
2200901995142,6cyclictest2206904-21kworker/2:309:58:392
2200897995131,16cyclictest2288228-21sh11:57:451
2200897995128,18cyclictest2236195-21cat10:10:011
2200897995127,19cyclictest2261533-21ssh11:02:211
2200897995124,22cyclictest2308727-21sh12:40:071
2200897995122,23cyclictest2224419-21ssh09:45:211
220089499516,36cyclictest2200898-21latency_hist07:10:010
2200901995043,4cyclictest2203474-21kworker/2:110:23:492
2200901995042,6cyclictest2203474-21kworker/2:109:37:112
2200901995042,5cyclictest2296514-21kworker/2:112:38:222
2200901995041,6cyclictest2203474-21kworker/2:111:07:592
2200901995033,13cyclictest275-21dbus-daemon09:55:012
2200897995036,11cyclictest2227731-21ssh09:52:231
2200897995032,15cyclictest2255229-21ssh10:49:361
2200897995021,24cyclictest2258428-21ssh10:55:481
2200906994919,9cyclictest37-21ksoftirqd/312:12:253
2200901994940,6cyclictest2203474-21kworker/2:111:11:462
2200901994940,5cyclictest2296514-21kworker/2:112:18:382
2200901994939,6cyclictest2206904-21kworker/2:309:19:552
2200901994935,11cyclictest2303842-21sed12:30:022
2200897994934,12cyclictest2286251-21ssh11:53:401
2200897994931,14cyclictest2282662-21ssh11:46:001
2200897994931,14cyclictest2267951-21ssh11:15:291
2200897994928,16cyclictest2275358-21ssh11:30:491
2200897994924,20cyclictest2206774-21latency_hist09:00:021
2200897994923,21cyclictest2301175-21ssh12:24:481
2200897994922,23cyclictest2217444-21sh09:30:581
220089499492,5cyclictest2202693-21munin-run07:45:000
2200906994833,10cyclictest1-21systemd07:55:013
2200906994831,12cyclictest275-21dbus-daemon10:45:003
2200906994824,10cyclictest37-21ksoftirqd/311:55:263
2200901994840,5cyclictest2206904-21kworker/2:309:21:352
2200901994840,5cyclictest2203474-21kworker/2:110:32:382
2200901994839,5cyclictest2203474-21kworker/2:110:37:322
2200901994834,10cyclictest3467602-21systemd-journal08:44:592
2200897994835,11cyclictest2223595-21ssh09:43:561
2200897994835,11cyclictest2223595-21ssh09:43:551
2200897994834,10cyclictest2202667-21cron07:45:001
2200897994832,12cyclictest2236544-21ssh10:10:271
2200897994831,13cyclictest2251342-21ssh10:41:101
2200897994827,17cyclictest2202921-21cron07:49:591
2200897994825,17cyclictest2204188-21modprobe08:11:541
2200897994820,21cyclictest2301588-21ssh12:25:121
2200906994723,8cyclictest37-21ksoftirqd/310:48:143
2200901994739,5cyclictest2203474-21kworker/2:110:51:052
2200901994739,5cyclictest2203474-21kworker/2:110:42:572
2200901994739,5cyclictest2193290-21kworker/2:007:43:112
2200901994724,17cyclictest275-21dbus-daemon11:55:012
2200901994712,7cyclictest131rcu_preempt11:41:262
2200897994729,14cyclictest2206079-21cron08:50:001
220089499472,7cyclictest2202920-21cron07:50:000
2200906994632,10cyclictest37-21ksoftirqd/312:20:113
220090699461,4cyclictest131rcu_preempt11:25:023
220090199464,3cyclictest131rcu_preempt10:47:292
2200901994636,6cyclictest2193290-21kworker/2:007:37:392
2200901994636,6cyclictest2193290-21kworker/2:007:37:382
2200901994615,9cyclictest30-21ksoftirqd/212:10:442
2200901994611,6cyclictest131rcu_preempt11:30:432
2200897994624,17cyclictest2202256-21latency_hist07:35:021
2200897994622,19cyclictest2205822-21sshd08:43:551
2200897994620,21cyclictest2207133-21latency_hist09:10:031
2200901994537,5cyclictest2287183-21kworker/2:012:03:172
2200901994537,5cyclictest2206904-21kworker/2:311:27:162
2200901994537,5cyclictest2206904-21kworker/2:311:23:392
2200901994537,5cyclictest2206904-21kworker/2:310:09:552
2200901994537,5cyclictest2203474-21kworker/2:109:40:252
2200901994537,5cyclictest2203474-21kworker/2:109:40:252
2200901994532,9cyclictest2203711-21cron08:05:002
2200901994524,13cyclictest309-21in:imuxsock08:10:032
2200897994533,8cyclictest275-21dbus-daemon08:55:011
2200897994529,13cyclictest2204454-21sshd08:18:471
2200897994528,13cyclictest1-21systemd08:25:591
2200906994422,4cyclictest2204484-21latency_hist08:20:023
2200901994437,4cyclictest2203474-21kworker/2:109:10:482
2200901994436,5cyclictest2206904-21kworker/2:309:47:342
2200901994431,9cyclictest30-21ksoftirqd/207:25:002
2200901994430,9cyclictest3467602-21systemd-journal08:40:002
2200901994427,11cyclictest309-21in:imuxsock07:45:022
2200901994424,9cyclictest2279603-21cron11:40:002
220090199442,3cyclictest131rcu_preempt07:35:002
2200901994418,20cyclictest275-21dbus-daemon09:05:012
2200901994412,29cyclictest291rcuc/212:25:032
2200897994428,12cyclictest1-21systemd08:40:001
2200897994422,18cyclictest2205307-21latency_hist08:35:021
2200897994422,18cyclictest2205307-21latency_hist08:35:021
2200901994335,5cyclictest2287183-21kworker/2:012:07:352
2200901994335,5cyclictest2206904-21kworker/2:309:25:262
2200901994335,5cyclictest2203474-21kworker/2:110:16:332
2200901994331,8cyclictest30-21ksoftirqd/208:50:012
2200901994330,9cyclictest309-21in:imuxsock09:32:542
2200901994328,11cyclictest30-21ksoftirqd/210:55:492
220089799433,31cyclictest2203679-21kworker/u8:108:25:021
2200897994326,13cyclictest2203716-21munin-run08:04:591
2200897994323,16cyclictest2202537-21cat07:40:021
2200897994323,16cyclictest2202537-21cat07:40:011
2200897994322,16cyclictest2201887-21modprobe07:30:011
2200897994322,16cyclictest2201887-21modprobe07:30:001
2200906994231,8cyclictest37-21ksoftirqd/312:10:023
2200906994229,10cyclictest37-21ksoftirqd/310:31:003
220090199425,26cyclictest2205303-21modprobe08:35:012
220090199425,26cyclictest2205303-21modprobe08:35:002
2200901994228,11cyclictest3467602-21systemd-journal08:09:312
2200901994226,12cyclictest30-21ksoftirqd/210:03:332
2200897994231,6cyclictest274-21cron07:20:001
2200897994229,10cyclictest2201073-21sshd07:13:421
2200897994224,15cyclictest364-21snmpd07:20:171
2200897994221,16cyclictest3467602-21systemd-journal07:53:451
220090199418,20cyclictest2206598-21modprobe09:00:002
2200901994132,6cyclictest2193290-21kworker/2:008:16:132
2200897994121,16cyclictest2203952-21sshd08:09:541
2200906994025,11cyclictest275-21dbus-daemon08:05:013
220090199406,6cyclictest131rcu_preempt07:15:002
2200897994030,6cyclictest2203484-21latency_hist08:00:021
2200906993928,8cyclictest37-21ksoftirqd/308:10:023
2200906993927,8cyclictest37-21ksoftirqd/309:10:033
2200906993926,10cyclictest37-21ksoftirqd/310:52:053
2200906993925,11cyclictest1-21systemd11:45:003
2200906993925,10cyclictest37-21ksoftirqd/310:16:533
2200906993913,5cyclictest131rcu_preempt09:30:073
2200901993931,5cyclictest2200487-21kworker/2:107:19:552
2200901993930,6cyclictest2206904-21kworker/2:310:13:452
2200901993928,9cyclictest30-21ksoftirqd/211:01:022
2200906993829,7cyclictest37-21ksoftirqd/309:44:093
2200906993829,7cyclictest37-21ksoftirqd/309:44:083
2200906993826,8cyclictest2277282-21cat11:35:023
2200906993817,15cyclictest275-21dbus-daemon07:35:013
2200901993822,12cyclictest309-21in:imuxsock09:05:022
2200897993816,19cyclictest2206890-21systemctl09:05:001
2200897993816,18cyclictest2201022-21cat07:10:021
220090699375,9cyclictest37-21ksoftirqd/307:40:033
2200906993721,13cyclictest2227730-21sh09:52:223
2200906993721,12cyclictest275-21dbus-daemon08:35:013
2200906993721,12cyclictest275-21dbus-daemon08:35:013
220090699371,6cyclictest131rcu_preempt09:05:003
2200906993627,6cyclictest2286707-21sh11:54:443
2200906993625,9cyclictest37-21ksoftirqd/310:23:063
220090199364,27cyclictest2205062-21grep08:30:002
2200901993623,9cyclictest2202037-21sed07:30:032
2200901993623,9cyclictest2202037-21sed07:30:032
2200901993616,16cyclictest275-21dbus-daemon07:55:002
2200906993524,8cyclictest37-21ksoftirqd/308:25:033
220090699342,6cyclictest2306097-21munin-run12:35:003
2200906993420,9cyclictest0-21swapper/307:40:013
2200906993420,9cyclictest0-21swapper/307:40:003
2200906993323,6cyclictest2205862-21grep08:45:013
220090199332,5cyclictest131rcu_preempt08:55:012
2200901993121,8cyclictest30-21ksoftirqd/208:25:012
2200901992812,13cyclictest2200905-21wc07:10:002
2200906992617,7cyclictest2200947-21latency_hist07:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional