You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-28 - 19:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat Mar 28, 2026 12:45:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26815359910397,4cyclictest2694375-21ssh09:25:020
268153599877,45cyclictest12-21ksoftirqd/012:10:010
2681540998624,33cyclictest2742776-21sh11:17:501
2681540997334,15cyclictest131rcu_preempt12:15:021
2681546997163,6cyclictest2687858-21ssh09:10:033
268154099712,12cyclictest48-21kcompactd011:05:031
2681545996941,19cyclictest3467602-21systemd-journal10:00:002
2681540996930,36cyclictest2730378-21sh10:49:111
2681540996929,36cyclictest2753189-21sh11:41:551
268154099691,55cyclictest0-21swapper/112:30:031
268154099691,55cyclictest0-21swapper/112:30:021
2681540996826,39cyclictest23-21ksoftirqd/110:40:331
2681540996826,39cyclictest23-21ksoftirqd/110:40:331
2681540996821,42cyclictest2758421-21sh11:54:311
268153599686,37cyclictest131rcu_preempt12:05:010
2681540996622,40cyclictest23-21ksoftirqd/109:13:491
268154099662,18cyclictest2768945-21ssh12:18:541
268154099662,18cyclictest2768945-21ssh12:18:541
268154099662,18cyclictest2731166-21sh10:50:311
268154099662,18cyclictest2731166-21sh10:50:311
268154099662,18cyclictest2722916-21ssh10:31:281
268154099662,18cyclictest2722916-21ssh10:31:281
268154099661,18cyclictest2746946-21sh11:27:271
268153599666,9cyclictest121ksoftirqd/010:37:330
268153599661,5cyclictest0-21swapper/010:01:490
2681540996528,33cyclictest2754679-21sh11:45:181
2681540996528,33cyclictest2754679-21sh11:45:171
268154099652,21cyclictest2718720-21rm10:21:481
2681540996428,32cyclictest2740212-21sh11:11:501
2681540996425,35cyclictest2699490-21sh09:37:111
2681540996425,35cyclictest2699490-21sh09:37:111
268154099642,12cyclictest2715365-21ssh10:14:231
268154099642,12cyclictest2715365-21ssh10:14:231
268154099641,17cyclictest2705444-21ssh09:50:481
268153599647,7cyclictest121ksoftirqd/011:08:590
268154099632,57cyclictest2763292-21ssh12:05:211
268154099632,20cyclictest2717567-21ssh10:19:221
268154099632,13cyclictest2704816-21ssh09:49:501
268154099631,9cyclictest2774219-21ssh12:30:451
268154099631,9cyclictest2760260-21ssh11:58:431
268154099631,9cyclictest2760260-21ssh11:58:431
268154099631,10cyclictest2744504-21ssh11:21:401
268154099631,10cyclictest2702247-21ssh09:43:371
268153599639,27cyclictest131rcu_preempt10:06:410
268153599638,24cyclictest12-21ksoftirqd/008:55:020
2681535996334,4cyclictest131rcu_preempt10:47:220
2681540996223,33cyclictest23-21ksoftirqd/110:03:141
268154099621,8cyclictest2748180-21ssh11:30:091
268154099621,7cyclictest2770147-21ssh12:21:191
268154099621,57cyclictest2707416-21ssh09:55:261
2681535996231,15cyclictest2768386-21ssh12:17:240
2681535996231,15cyclictest2768386-21ssh12:17:240
2681535996210,5cyclictest131rcu_preempt12:21:560
268154599611,40cyclictest2730836-21ssh10:50:012
268154099613,12cyclictest2684598-21munin-run08:10:001
268154099612,6cyclictest2685659-21munin-run08:30:011
268154099611,8cyclictest2776274-21ssh12:35:341
268154099611,8cyclictest2762343-21ssh12:03:281
268154099611,8cyclictest2750948-21ssh11:36:431
268154099611,8cyclictest2693822-21ssh09:24:141
268154099611,7cyclictest2720395-21ssh10:25:351
268154099611,7cyclictest2711948-21ssh10:05:551
268154099611,7cyclictest2694507-21ssh09:25:191
268154099611,7cyclictest2691345-21ssh09:18:301
268154099611,6cyclictest2735587-21ssh11:00:551
268153599617,8cyclictest12-21ksoftirqd/011:01:150
2681545996039,16cyclictest3467602-21systemd-journal08:15:002
2681540996028,27cyclictest2682275-21munin-run07:25:001
268154099602,54cyclictest2687481-21munin-run09:05:001
268154099602,13cyclictest2733202-21ssh10:55:211
268154099601,7cyclictest2724978-21ssh10:36:181
268154099601,15cyclictest2696889-21sh09:30:581
268153599607,9cyclictest121ksoftirqd/009:31:130
268153599606,33cyclictest0-21swapper/009:51:160
268153599605,13cyclictest12-21ksoftirqd/009:21:150
268153599601,9cyclictest12-21ksoftirqd/010:16:450
268154099592,5cyclictest2681735-21munin-run07:14:591
268154099591,8cyclictest1-21systemd07:30:011
2681535995950,6cyclictest151rcuc/011:10:040
268153599594,28cyclictest12-21ksoftirqd/010:14:010
268153599594,28cyclictest12-21ksoftirqd/010:14:010
2681535995918,36cyclictest16850irq/134-800000011:31:530
2681535995915,4cyclictest2727572-21ssh10:42:230
2681535995915,4cyclictest2727572-21ssh10:42:220
268154099583,49cyclictest2683801-21cron07:55:001
268154099582,5cyclictest2687199-21munin-run09:00:011
268154099581,6cyclictest2685084-21sshd08:17:161
268154099581,13cyclictest364-21snmpd08:21:561
2681535995822,31cyclictest2700626-21latency_hist09:40:020
2681535995822,31cyclictest2700626-21latency_hist09:40:020
2681535995817,12cyclictest12-21ksoftirqd/009:11:110
2681546995715,13cyclictest37-21ksoftirqd/311:10:013
2681545995748,6cyclictest2687495-21kworker/2:309:50:022
2681545995746,7cyclictest2674628-21kworker/2:212:30:022
2681545995746,7cyclictest2674628-21kworker/2:212:30:022
2681545995746,6cyclictest2674628-21kworker/2:211:54:412
2681545995744,9cyclictest2674628-21kworker/2:208:50:012
268154099572,5cyclictest2686935-21munin-run08:55:011
268154099571,12cyclictest2681969-21sshd07:17:161
2681535995730,23cyclictest16850irq/134-800000011:51:140
2681535995726,27cyclictest12-21ksoftirqd/009:50:020
2681535995713,15cyclictest2722428-21ssh10:30:130
2681535995713,15cyclictest2722428-21ssh10:30:130
268154099562,8cyclictest2683311-21munin-run07:45:001
268154099561,49cyclictest364-21snmpd08:47:421
268153599568,9cyclictest12-21ksoftirqd/012:33:170
2681535995648,5cyclictest16950irq/224-800000012:40:020
2681545995545,6cyclictest2687495-21kworker/2:310:24:432
268154099553,45cyclictest3467602-21systemd-journal08:00:011
268154099552,48cyclictest0-21swapper/109:08:041
268154099551,4cyclictest2686127-21sshd08:37:161
268153599556,7cyclictest121ksoftirqd/012:26:480
268153599556,7cyclictest121ksoftirqd/012:26:470
268153599554,12cyclictest12-21ksoftirqd/010:53:090
268153599554,12cyclictest12-21ksoftirqd/010:53:090
2681545995445,5cyclictest2674628-21kworker/2:211:40:112
2681545995443,7cyclictest2687495-21kworker/2:309:26:492
2681545995435,15cyclictest1-21systemd08:10:012
268154099541,22cyclictest2686437-21true08:45:011
268153599548,5cyclictest131rcu_preempt10:21:420
268153599547,4cyclictest131rcu_preempt09:18:420
2681535995435,9cyclictest19350irq/34-mmc111:45:020
2681535995427,11cyclictest2765590-21ssh12:10:500
268154699531,24cyclictest0-21swapper/312:25:013
2681545995343,6cyclictest2687495-21kworker/2:309:25:022
268154099532,17cyclictest2685907-21systemd-run08:35:001
268154099531,4cyclictest364-21snmpd07:47:501
268153599535,8cyclictest12-21ksoftirqd/011:20:350
268153599535,6cyclictest0-21swapper/011:16:540
2681535995344,5cyclictest16950irq/224-800000009:00:010
2681535995325,13cyclictest2734312-21ssh10:58:170
2681545995243,5cyclictest2674628-21kworker/2:211:48:572
2681545995243,5cyclictest2674628-21kworker/2:211:48:572
2681545995242,6cyclictest2687495-21kworker/2:310:40:002
2681545995237,11cyclictest1-21systemd09:20:012
2681545995235,14cyclictest30-21ksoftirqd/211:20:032
268154099522,7cyclictest364-21snmpd08:10:561
268154099522,3cyclictest364-21snmpd08:02:521
2681535995216,17cyclictest16850irq/134-800000009:42:330
268153599521,37cyclictest2747296-21ssh11:28:240
2681545995143,5cyclictest2674628-21kworker/2:210:10:222
2681545995143,5cyclictest2674628-21kworker/2:210:10:222
2681545995142,5cyclictest2687495-21kworker/2:309:10:022
2681545995141,6cyclictest2674628-21kworker/2:208:40:002
2681545995141,6cyclictest2671945-21kworker/2:007:40:002
2681545995139,7cyclictest2748038-21latency_hist11:30:022
2681545995133,13cyclictest275-21dbus-daemon07:50:012
268154099512,6cyclictest1-21systemd07:35:011
268154099512,4cyclictest0-21swapper/107:37:161
2681535995134,15cyclictest0-21swapper/011:50:020
2681535995134,15cyclictest0-21swapper/011:50:020
2681535995132,14cyclictest2709218-21systemctl10:00:000
268153599512,13cyclictest12-21ksoftirqd/008:35:000
268154699501,9cyclictest37-21ksoftirqd/312:00:013
268154699501,9cyclictest37-21ksoftirqd/312:00:013
2681545995042,5cyclictest2683575-21kworker/2:108:20:012
2681545995042,5cyclictest2674628-21kworker/2:211:36:162
2681545995042,5cyclictest2674628-21kworker/2:208:55:092
2681545995042,5cyclictest2674628-21kworker/2:208:35:002
2681545995041,6cyclictest2674628-21kworker/2:212:17:072
2681545995041,6cyclictest2674628-21kworker/2:212:17:072
2681545995041,5cyclictest2674628-21kworker/2:212:32:312
268153599507,7cyclictest12-21ksoftirqd/007:50:010
268154699496,6cyclictest131rcu_preempt09:31:413
2681545994941,5cyclictest2687495-21kworker/2:310:02:032
2681545994941,5cyclictest2687495-21kworker/2:309:50:242
2681545994941,5cyclictest2674628-21kworker/2:211:02:292
2681545994940,6cyclictest2674628-21kworker/2:208:55:012
268153599499,8cyclictest12-21ksoftirqd/010:26:270
268153599492,9cyclictest12-21ksoftirqd/008:15:000
268154699489,4cyclictest131rcu_preempt09:15:223
2681545994840,5cyclictest2687495-21kworker/2:310:33:452
2681545994840,5cyclictest2687495-21kworker/2:310:33:452
2681545994840,5cyclictest2687495-21kworker/2:309:37:352
2681545994840,5cyclictest2687495-21kworker/2:309:37:352
2681545994840,5cyclictest2674628-21kworker/2:212:23:212
2681545994840,5cyclictest2674628-21kworker/2:212:09:552
2681545994840,5cyclictest2674628-21kworker/2:212:00:452
2681545994840,5cyclictest2674628-21kworker/2:211:14:022
2681545994839,6cyclictest2674628-21kworker/2:211:31:532
2681545994839,5cyclictest2674628-21kworker/2:211:59:462
2681545994839,5cyclictest2674628-21kworker/2:211:59:462
2681545994839,5cyclictest2674628-21kworker/2:208:45:012
2681535994820,13cyclictest2759079-21ssh11:55:340
2681535994820,13cyclictest2759079-21ssh11:55:340
2681546994731,11cyclictest3467602-21systemd-journal11:30:003
2681545994741,4cyclictest2674628-21kworker/2:211:21:412
2681545994740,5cyclictest2687495-21kworker/2:310:42:292
2681545994740,5cyclictest2687495-21kworker/2:310:42:292
2681545994739,5cyclictest2687495-21kworker/2:310:05:292
2681545994739,5cyclictest2687495-21kworker/2:309:42:292
2681545994739,5cyclictest2674628-21kworker/2:210:58:092
2681545994739,5cyclictest2674628-21kworker/2:210:18:572
2681545994739,4cyclictest2687495-21kworker/2:309:35:012
2681545994738,5cyclictest2674628-21kworker/2:212:38:402
268153599479,9cyclictest12-21ksoftirqd/011:35:450
2681535994738,7cyclictest12-21ksoftirqd/008:45:020
268153599471,14cyclictest12-21ksoftirqd/009:05:020
268154699468,5cyclictest131rcu_preempt12:10:033
268154699465,7cyclictest131rcu_preempt10:21:573
2681546994636,7cyclictest37-21ksoftirqd/309:55:013
2681546994635,7cyclictest37-21ksoftirqd/310:30:003
2681546994632,10cyclictest0-21swapper/311:40:023
2681546994613,8cyclictest131rcu_preempt09:00:013
2681545994638,5cyclictest2687495-21kworker/2:310:26:122
2681545994638,5cyclictest2674628-21kworker/2:212:13:282
268153599461,5cyclictest2685114-21modprobe08:20:000
2681535994610,6cyclictest131rcu_preempt07:10:020
2681546994529,11cyclictest37-21ksoftirqd/309:40:003
2681546994529,11cyclictest37-21ksoftirqd/309:40:003
2681546994529,11cyclictest275-21dbus-daemon08:50:013
268154699452,14cyclictest37-21ksoftirqd/308:19:593
2681546994519,18cyclictest37-21ksoftirqd/310:35:013
2681546994519,18cyclictest37-21ksoftirqd/310:35:013
2681546994517,16cyclictest2687724-21diskmemload11:50:023
2681546994517,16cyclictest2687724-21diskmemload11:50:023
2681545994537,5cyclictest2687495-21kworker/2:311:05:462
2681545994537,5cyclictest2687495-21kworker/2:310:51:192
2681545994537,5cyclictest2687495-21kworker/2:310:51:192
2681545994537,5cyclictest2687495-21kworker/2:309:14:422
2681545994536,6cyclictest2674628-21kworker/2:207:23:482
2681545994536,5cyclictest2683575-21kworker/2:108:21:362
2681545994535,6cyclictest2674628-21kworker/2:208:00:012
268154099451,3cyclictest0-21swapper/107:10:021
2681535994540,3cyclictest150-21pr/ttyS207:15:020
2681535994522,17cyclictest275-21dbus-daemon07:55:010
268153599452,12cyclictest2687731-21grep09:10:000
268154699445,9cyclictest37-21ksoftirqd/309:26:383
268154699443,4cyclictest131rcu_preempt11:20:503
2681546994434,6cyclictest37-21ksoftirqd/312:15:323
2681546994434,6cyclictest37-21ksoftirqd/312:15:313
2681546994432,8cyclictest3467602-21systemd-journal12:35:023
268154699442,6cyclictest131rcu_preempt10:16:163
2681545994436,5cyclictest2671945-21kworker/2:007:15:022
268153599446,10cyclictest12-21ksoftirqd/007:35:010
2681546994331,9cyclictest37-21ksoftirqd/310:59:253
2681546994330,9cyclictest3467602-21systemd-journal09:20:033
2681546994329,11cyclictest3467602-21systemd-journal11:10:183
2681546994328,10cyclictest37-21ksoftirqd/311:03:563
2681546994327,11cyclictest309-21in:imuxsock07:22:173
268154699431,7cyclictest0-21swapper/311:31:023
2681546994317,18cyclictest3467602-21systemd-journal08:10:003
2681546994317,17cyclictest37-21ksoftirqd/311:18:543
2681546994312,6cyclictest131rcu_preempt09:05:013
268154699431,18cyclictest2778004-21ssh12:40:003
268153599437,11cyclictest12-21ksoftirqd/007:45:010
2681535994335,5cyclictest16950irq/224-800000008:40:010
2681546994233,6cyclictest37-21ksoftirqd/309:45:003
2681546994232,7cyclictest37-21ksoftirqd/312:05:003
2681546994232,6cyclictest37-21ksoftirqd/312:12:543
268154699423,11cyclictest37-21ksoftirqd/311:42:533
268154699422,5cyclictest131rcu_preempt09:45:113
268154699421,9cyclictest0-21swapper/311:54:593
268154699421,5cyclictest0-21swapper/310:14:373
268154699421,5cyclictest0-21swapper/310:14:373
2681545994212,6cyclictest131rcu_preempt07:33:152
268153599422,6cyclictest2683017-21cron07:40:000
268153599422,36cyclictest2682249-21sshd07:21:160
268153599422,34cyclictest2684070-21latency_hist08:00:000
268153599422,11cyclictest12-21ksoftirqd/008:22:160
2681546994133,6cyclictest37-21ksoftirqd/310:38:203
2681546994132,6cyclictest37-21ksoftirqd/310:53:343
2681546994132,6cyclictest37-21ksoftirqd/310:53:343
268154699413,10cyclictest37-21ksoftirqd/310:45:023
268154699413,10cyclictest37-21ksoftirqd/310:45:023
268154699411,9cyclictest0-21swapper/309:55:583
268154699411,8cyclictest0-21swapper/312:28:083
268154699411,8cyclictest0-21swapper/312:28:083
268154699411,7cyclictest0-21swapper/310:49:293
268154699411,10cyclictest2711587-21ssh10:05:033
2681545994031,5cyclictest2683575-21kworker/2:108:30:012
2681545994030,6cyclictest2671945-21kworker/2:007:20:012
268153599406,8cyclictest12-21ksoftirqd/008:48:000
2681535994030,8cyclictest12-21ksoftirqd/007:20:020
268153599402,9cyclictest0-21swapper/008:29:100
268154699395,14cyclictest37-21ksoftirqd/308:00:003
2681546993926,9cyclictest2681803-21sed07:15:023
2681546993923,11cyclictest3467602-21systemd-journal07:55:003
268154699391,7cyclictest0-21swapper/309:13:443
268154699391,6cyclictest0-21swapper/310:02:503
2681545993918,15cyclictest275-21dbus-daemon07:30:012
2681545993911,5cyclictest131rcu_preempt07:42:482
268153599394,8cyclictest12-21ksoftirqd/008:09:050
268153599394,8cyclictest12-21ksoftirqd/007:26:160
268153599392,7cyclictest12-21ksoftirqd/008:03:160
2681546993814,6cyclictest131rcu_preempt07:35:033
2681545993823,11cyclictest3467602-21systemd-journal08:05:002
268154599382,30cyclictest309-21in:imuxsock07:55:012
268154699379,19cyclictest2683776-21kworker/u8:008:30:023
2681546993728,7cyclictest37-21ksoftirqd/308:45:013
2681545993720,12cyclictest275-21dbus-daemon09:05:012
2681546993624,7cyclictest2681982-21cron07:20:003
2681546993622,11cyclictest37-21ksoftirqd/307:45:003
2681546993618,13cyclictest3467602-21systemd-journal08:35:023
268154699353,5cyclictest131rcu_preempt08:34:593
2681546993526,6cyclictest37-21ksoftirqd/308:55:023
268154699352,5cyclictest131rcu_preempt07:27:163
268154699335,5cyclictest131rcu_preempt07:37:333
2681546993324,7cyclictest37-21ksoftirqd/307:10:013
2681546993323,7cyclictest37-21ksoftirqd/307:50:013
2681546993220,9cyclictest37-21ksoftirqd/308:01:163
268154699321,5cyclictest131rcu_preempt08:13:513
2681546993016,11cyclictest37-21ksoftirqd/308:22:553
2681545992918,8cyclictest30-21ksoftirqd/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional