You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 02:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Fri Apr 24, 2026 12:45:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2532555998735,13cyclictest2538715-21cron09:10:000
2532555998735,13cyclictest2538715-21cron09:10:000
2532555998331,16cyclictest2594683-21cat11:20:010
2532555998331,16cyclictest2594683-21cat11:20:000
2532555998129,14cyclictest2552580-21ssh09:41:570
2532555997731,12cyclictest2603336-21ssh11:40:010
2532560997532,15cyclictest2542848-21ssh09:19:451
2532555997533,22cyclictest2569038-21ssh10:20:030
2532560997433,14cyclictest2550192-21sh09:36:261
2532555997470,2cyclictest0-21swapper/012:00:020
2532555997426,15cyclictest2558884-21ssh09:56:440
2532564997310,4cyclictest131rcu_preempt09:30:012
2532560997325,15cyclictest2615089-21ssh12:06:491
253256999723,7cyclictest131rcu_preempt08:55:033
2532560997124,16cyclictest2561811-21ssh10:03:391
2532555997166,3cyclictest0-21swapper/011:35:020
2532560997017,20cyclictest2628583-21ssh12:38:161
2532555996926,11cyclictest2589385-21ssh11:07:260
2532555996926,11cyclictest2589385-21ssh11:07:250
2532569996819,43cyclictest275-21dbus-daemon08:15:003
2532560996824,12cyclictest2538730-21diskmemload10:19:411
2532569996515,13cyclictest275-21dbus-daemon10:05:003
2532555996555,6cyclictest16950irq/224-800000010:10:010
2532569996418,42cyclictest275-21dbus-daemon08:50:003
253256499643,33cyclictest2582003-21kworker/u8:411:10:032
253256499643,33cyclictest2582003-21kworker/u8:411:10:022
2532560996451,10cyclictest2626049-21ssh12:32:191
2532560996450,10cyclictest2593072-21ssh11:15:451
2532560996450,10cyclictest2593072-21ssh11:15:441
2532560996448,13cyclictest2598800-21ssh11:29:221
2532560996448,13cyclictest2598800-21ssh11:29:221
2532560996446,15cyclictest2558974-21ssh09:56:581
2532560996425,12cyclictest2590024-21ssh11:09:091
2532560996425,12cyclictest2590024-21ssh11:09:081
2532555996445,12cyclictest16950irq/224-800000012:35:010
2532569996322,36cyclictest2545557-21sh09:25:313
253256999632,15cyclictest2538752-21/usr/sbin/munin09:10:023
253256999632,15cyclictest2538752-21/usr/sbin/munin09:10:013
2532569996321,38cyclictest2557600-21sh09:54:033
2532569996319,40cyclictest3467602-21systemd-journal12:24:313
2532560996350,10cyclictest2556169-21ssh09:50:171
2532560996347,13cyclictest2576789-21ssh10:38:171
2532560996347,12cyclictest2596284-21ssh11:23:231
2532560996347,12cyclictest2584782-21ssh10:56:371
2532560996346,13cyclictest2620926-21ssh12:20:081
2532560996346,13cyclictest2563839-21ssh10:08:161
2532560996345,14cyclictest2601590-21ssh11:35:311
2532560996344,15cyclictest2544188-21ssh09:22:381
2532560996315,17cyclictest2575031-21ssh10:34:181
2532555996310,15cyclictest0-21swapper/010:50:020
2532569996233,21cyclictest309-21in:imuxsock12:20:003
253256999623,14cyclictest48-21kcompactd010:30:283
253256999622,14cyclictest2575664-21sshd10:35:243
253256499623,43cyclictest0-21swapper/210:10:022
2532560996248,11cyclictest2587311-21ssh11:02:321
2532560996245,14cyclictest2552690-21ssh09:42:141
2532560996245,13cyclictest2619038-21ssh12:15:581
2532560996245,13cyclictest2616857-21ssh12:10:511
2532560996245,13cyclictest2606628-21ssh11:47:211
2532560996245,13cyclictest2591027-21ssh11:11:081
2532560996245,13cyclictest2578091-21ssh10:41:041
2532560996245,13cyclictest2564894-21ssh10:10:251
2532560996245,13cyclictest2546222-21ssh09:27:151
2532560996244,14cyclictest2608402-21ssh11:51:221
2532560996225,14cyclictest150-21pr/ttyS208:10:551
253256999613,16cyclictest2560065-21ssh09:59:503
253256999611,15cyclictest364-21snmpd09:14:183
2532564996152,5cyclictest2526266-21kworker/2:111:44:592
2532564996111,20cyclictest2618112-21sh12:14:082
2532560996146,12cyclictest2610221-21ssh11:55:261
2532560996146,12cyclictest2582130-21ssh10:50:241
2532560996146,11cyclictest2539899-21ssh09:12:471
2532560996145,14cyclictest2570320-21ssh10:23:131
2532560996145,13cyclictest2603644-21ssh11:40:091
2532560996145,13cyclictest2554713-21ssh09:47:001
2532560996145,13cyclictest2548144-21ssh09:31:461
2532560996145,13cyclictest2548144-21ssh09:31:451
2532560996145,12cyclictest2623751-21ssh12:26:531
2532560996144,14cyclictest2599348-21ssh11:30:141
2532560996142,15cyclictest2612398-21ssh12:00:301
2532560996142,15cyclictest2612398-21ssh12:00:291
2532555996135,18cyclictest2537148-21cron08:40:010
253256999602,16cyclictest2578011-21ssh10:40:523
253256999602,16cyclictest2536168-21munin-run08:20:023
253256999602,15cyclictest48-21kcompactd012:35:153
253256999602,15cyclictest2535057-21sshd07:59:253
253256999602,15cyclictest2535057-21sshd07:59:243
253256999602,14cyclictest2534576-21munin-run07:49:593
2532569996017,13cyclictest2596969-21cat11:25:023
2532564996021,7cyclictest30-21ksoftirqd/210:50:442
2532564996016,7cyclictest30-21ksoftirqd/210:39:342
2532560996047,10cyclictest2572191-21ssh10:27:271
2532560996044,13cyclictest2579964-21ssh10:45:171
2532560996037,18cyclictest2537661-21sshd08:47:241
253256999593,19cyclictest2586102-21munin-run11:00:003
253256999593,16cyclictest2614252-21sh12:05:043
253256999593,16cyclictest2614252-21sh12:05:033
253256999592,16cyclictest3467602-21systemd-journal10:25:013
2532569995921,34cyclictest37-21ksoftirqd/311:25:123
2532569995921,34cyclictest37-21ksoftirqd/311:25:113
2532569995921,34cyclictest2580199-21sh10:45:513
2532569995921,34cyclictest2538730-21diskmemload11:46:423
253256999591,17cyclictest2587254-21sshd11:02:253
253256999591,16cyclictest364-21snmpd08:39:023
2532564995917,9cyclictest30-21ksoftirqd/210:26:062
2532560995946,10cyclictest2534010-21sshd07:35:231
2532560995944,11cyclictest2538454-21sshd09:04:241
2532560995943,13cyclictest2535575-21sshd08:05:241
2532560995943,13cyclictest2535575-21sshd08:05:241
253256999587,46cyclictest1-21systemd11:20:023
253256999587,46cyclictest1-21systemd11:20:013
2532569995819,34cyclictest2533183-21sort07:20:023
2532564995815,19cyclictest2592323-21sh11:14:312
2532564995811,9cyclictest30-21ksoftirqd/210:55:452
2532560995844,10cyclictest2538691-21sshd09:05:241
2532560995844,10cyclictest2538691-21sshd09:05:231
2532560995844,10cyclictest2538177-21sshd08:59:361
253256999573,14cyclictest2590728-21sshd11:10:243
2532569995726,26cyclictest2553422-21sh09:44:093
2532569995724,29cyclictest2617687-21sh12:13:003
253256999572,15cyclictest2563006-21ssh10:06:093
253256999572,14cyclictest2616298-21munin-run12:10:003
2532569995720,33cyclictest37-21ksoftirqd/309:39:443
2532569995720,33cyclictest2538730-21diskmemload10:54:463
2532569995719,34cyclictest2602943-21sh11:39:003
2532569995717,4cyclictest1-21systemd07:25:013
2532569995715,7cyclictest2549541-21latency_hist09:35:033
2532569995715,7cyclictest2549541-21latency_hist09:35:023
253256999571,15cyclictest2624028-21ssh12:27:343
2532560995744,10cyclictest2537139-21sshd08:39:241
2532560995744,10cyclictest2536638-21sshd08:29:251
2532560995743,10cyclictest2534338-21munin-run07:45:011
2532560995736,16cyclictest364-21snmpd07:29:201
2532555995735,14cyclictest2551564-21cron09:40:000
253255599573,48cyclictest2533024-21cut07:20:000
2532555995726,15cyclictest2564861-21ssh10:10:190
253256999562,17cyclictest2538204-21munin-run08:59:593
253256999562,16cyclictest2536718-21grep08:30:013
2532569995619,33cyclictest2565165-21sh10:11:103
2532569995615,4cyclictest2567223-21sh10:15:533
253256999561,17cyclictest2533544-21munin-run07:30:003
2532560995644,9cyclictest2535340-21sshd08:04:241
2532560995644,9cyclictest2533270-21sshd07:24:201
253256999552,18cyclictest2608232-21ssh11:50:563
2532569995511,37cyclictest0-21swapper/312:35:003
253256999551,11cyclictest2600209-21rm11:32:263
2532564995511,7cyclictest30-21ksoftirqd/210:32:222
2532560995543,9cyclictest3467602-21systemd-journal07:53:341
2532569995428,22cyclictest2588854-21sh11:05:593
2532569995428,22cyclictest2588854-21sh11:05:593
253256999542,12cyclictest3467602-21systemd-journal07:39:593
2532569995415,11cyclictest3467602-21systemd-journal07:45:003
2532569995414,6cyclictest0-21swapper/309:48:543
2532569995414,4cyclictest2537434-21latency_hist08:45:003
253256999541,15cyclictest2610726-21ssh11:56:483
2532560995440,10cyclictest2536922-21munin-run08:35:001
2532560995439,11cyclictest364-21snmpd07:13:341
2532560995436,14cyclictest364-21snmpd07:31:511
2532555995434,5cyclictest16850irq/134-800000011:00:020
2532555995426,12cyclictest2615673-21ssh12:08:230
2532555995425,11cyclictest12-21ksoftirqd/012:19:590
253256999532,15cyclictest2604568-21ssh11:42:313
253256999532,13cyclictest2535719-21latency_hist08:10:033
253256999532,13cyclictest2535719-21latency_hist08:10:023
253256999531,15cyclictest2571871-21ssh10:26:413
2532560995342,8cyclictest48-21kcompactd008:16:261
2532560995340,10cyclictest2535123-21munin-run08:00:021
2532560995340,10cyclictest2535123-21munin-run08:00:011
2532560995332,17cyclictest2534557-21sshd07:49:251
253255599534,39cyclictest2534089-21unin-run07:40:000
2532555995337,12cyclictest3467602-21systemd-journal10:25:000
253255599533,41cyclictest2588379-21latency_hist11:05:020
2532555995324,13cyclictest2628129-21ssh12:37:040
253255599532,36cyclictest2540174-21ssh09:13:260
253255599532,28cyclictest2609907-21latency_hist11:55:020
2532555995314,36cyclictest2545114-21cron09:25:000
253256999522,14cyclictest2536931-21wc08:35:013
253256499526,15cyclictest2586376-21ssh11:00:052
2532560995229,19cyclictest2536542-21latency_hist08:25:021
2532560995229,19cyclictest2536542-21latency_hist08:25:021
253255599524,20cyclictest2581754-21munin-plugin-st10:50:010
253255599524,10cyclictest3467602-21systemd-journal08:20:000
2532555995234,12cyclictest12-21ksoftirqd/009:20:000
2532555995231,16cyclictest2611914-21kworker/u8:212:30:010
2532555995225,11cyclictest2607335-21ssh11:49:120
2532555995224,12cyclictest2577362-21ssh10:39:470
2532555995223,10cyclictest12-21ksoftirqd/009:34:040
2532555995223,10cyclictest12-21ksoftirqd/009:34:040
2532569995124,25cyclictest37-21ksoftirqd/309:21:503
253256999512,14cyclictest2541724-21ssh09:16:523
2532564995130,18cyclictest2616256-21ssh12:09:592
2532564995112,15cyclictest0-21swapper/212:30:002
2532560995139,9cyclictest2537562-21cat08:45:031
2532555995137,11cyclictest1-21systemd08:45:000
253256999501,15cyclictest364-21snmpd08:24:383
253256999501,15cyclictest364-21snmpd08:24:373
253256499501,27cyclictest0-21swapper/207:55:022
2532560995041,5cyclictest2532974-21cron07:17:001
253255599502,44cyclictest0-21swapper/010:41:530
2532555995023,12cyclictest2574726-21ssh10:33:340
2532555995022,12cyclictest2612938-21ssh12:01:560
2532555995022,12cyclictest2612938-21ssh12:01:550
253256499497,12cyclictest0-21swapper/208:55:022
2532564994930,15cyclictest2535837-21sshd08:12:252
2532555994939,6cyclictest16850irq/134-800000009:46:260
2532555994919,14cyclictest2557823-21ssh09:54:350
253256999486,9cyclictest2532744-21modprobe07:15:003
2532564994840,5cyclictest2545182-21kworker/2:309:55:272
2532564994831,14cyclictest2556277-21ssh09:50:322
2532564994831,13cyclictest2611589-21ssh11:59:012
2532564994830,14cyclictest2599589-21ssh11:30:502
2532564994829,16cyclictest2562120-21sshd10:04:252
2532564994829,15cyclictest2541892-21ssh09:17:152
2532564994811,33cyclictest275-21dbus-daemon08:10:012
2532564994811,33cyclictest275-21dbus-daemon08:10:012
2532560994832,12cyclictest364-21snmpd08:54:421
253255599484,39cyclictest2573147-21modprobe10:29:590
2532555994835,10cyclictest12-21ksoftirqd/011:26:460
2532555994835,10cyclictest12-21ksoftirqd/011:26:450
2532555994834,11cyclictest12-21ksoftirqd/011:45:000
2532555994829,15cyclictest2618407-21ssh12:15:010
2532555994822,11cyclictest2547206-21ssh09:29:570
2532564994739,5cyclictest2614186-21kworker/2:012:15:592
2532564994732,12cyclictest3467602-21systemd-journal07:26:242
2532564994732,11cyclictest3467602-21systemd-journal09:43:242
2532564994730,14cyclictest2543743-21ssh09:21:262
2532564994730,14cyclictest2535569-21cat08:05:032
2532564994730,13cyclictest2627716-21ssh12:36:062
2532564994730,13cyclictest2549677-21ssh09:35:132
2532564994730,13cyclictest2547882-21ssh09:31:012
2532564994730,13cyclictest2547882-21ssh09:31:002
2532564994726,17cyclictest2567119-21ssh10:15:392
2532564994715,7cyclictest131rcu_preempt11:50:032
2532564994715,23cyclictest2579649-21ssh10:45:022
2532564994713,7cyclictest2565448-21ssh10:11:512
253255599473,37cyclictest311-21rs:main0
2532555994733,10cyclictest12-21ksoftirqd/007:55:010
2532555994732,9cyclictest12-21ksoftirqd/012:25:020
2532555994727,15cyclictest3467602-21systemd-journal08:05:000
2532555994725,16cyclictest275-21dbus-daemon08:10:000
2532555994725,16cyclictest275-21dbus-daemon08:10:000
253256999462,17cyclictest2535500-21latency_hist08:05:013
2532564994632,12cyclictest2625236-21ssh12:30:172
2532564994631,13cyclictest2597461-21ssh11:26:012
2532564994631,13cyclictest2597461-21ssh11:26:002
2532564994631,11cyclictest2601998-21ssh11:36:302
2532564994630,13cyclictest2539036-21ssh09:10:322
2532564994629,14cyclictest2608086-21ssh11:50:342
2532564994629,14cyclictest2595074-21ssh11:20:232
2532555994631,11cyclictest309-21in:imuxsock07:45:020
2532555994630,12cyclictest12-21ksoftirqd/008:50:010
2532555994630,12cyclictest1-21systemd08:15:010
2532555994629,12cyclictest3467602-21systemd-journal08:55:020
2532555994627,15cyclictest2560645-21ssh10:00:340
253256999452,14cyclictest2538573-21cat09:05:013
2532564994538,4cyclictest2526266-21kworker/2:107:48:512
2532564994532,10cyclictest2538457-21sshd09:04:382
2532564994531,12cyclictest2554577-21ssh09:46:412
2532564994531,11cyclictest2569130-21ssh10:20:042
2532564994530,13cyclictest2592914-21ssh11:15:242
2532564994530,13cyclictest2592914-21ssh11:15:242
2532564994529,13cyclictest2612453-21ssh12:00:372
2532564994529,13cyclictest2612453-21ssh12:00:362
2532564994529,12cyclictest2579954-21ssh10:45:162
2532564994528,14cyclictest2620891-21ssh12:20:052
2532564994528,13cyclictest2533776-21sshd07:34:222
2532555994523,16cyclictest3467602-21systemd-journal08:25:020
2532555994523,16cyclictest3467602-21systemd-journal08:25:010
2532555994523,15cyclictest275-21dbus-daemon08:30:000
2532560994427,14cyclictest2532548-21cyclictest07:10:011
2532555994426,15cyclictest2596407-21ssh11:23:420
2532564994331,9cyclictest2537123-21sshd08:35:242
2532564994330,10cyclictest2536378-21sshd08:22:252
2532564994330,10cyclictest2536378-21sshd08:22:242
2532564994329,9cyclictest48-21kcompactd009:09:072
2532564994329,9cyclictest48-21kcompactd009:09:072
2532564994329,10cyclictest2537438-21munin-run08:45:002
2532555994336,4cyclictest16950irq/224-800000008:00:000
2532555994336,4cyclictest16950irq/224-800000008:00:000
2532555994323,15cyclictest275-21dbus-daemon07:45:010
2532564994233,6cyclictest2537689-21munin-plugin-st08:50:012
2532564994229,9cyclictest2535091-21munin-run08:00:002
2532564994229,9cyclictest2535091-21munin-run08:00:002
2532564994228,9cyclictest48-21kcompactd008:58:342
2532564994228,11cyclictest2536892-21sshd08:34:322
2532564994223,15cyclictest364-21snmpd08:28:462
2532555994227,11cyclictest12-21ksoftirqd/009:05:020
2532555994219,15cyclictest275-21dbus-daemon08:35:010
253256999412,15cyclictest2533856-21grep07:35:033
2532564994129,9cyclictest2534290-21sshd07:40:242
2532564994128,10cyclictest2532983-21sshd07:17:242
2532555994134,4cyclictest16950irq/224-800000007:15:020
2532555994130,8cyclictest0-21swapper/007:10:010
253255599403,5cyclictest0-21swapper/008:55:010
253255599401,35cyclictest0-21swapper/007:35:000
2532564993932,4cyclictest2533303-21/usr/sbin/munin07:25:022
2532564993927,10cyclictest30-21ksoftirqd/208:20:002
2532569993815,5cyclictest2532656-21latency_hist07:10:023
253256999366,4cyclictest131rcu_preempt07:55:013
2532564993623,9cyclictest364-21snmpd07:35:242
2532564993623,10cyclictest364-21snmpd07:10:392
253255599361,31cyclictest0-21swapper/007:25:020
2532555993521,11cyclictest12-21ksoftirqd/007:30:010
253256499299,17cyclictest2532576-21sed07:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional