You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 14:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sun May 10, 2026 12:43:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3281800995242,6cyclictest3435208-21basename11:40:002
3281800995040,6cyclictest3428651-21ntpq11:30:572
3281798995042,5cyclictest3383646-21systemd-run10:35:010
3281798994841,4cyclictest3372476-21systemd-run10:20:010
3281798994837,8cyclictest3345903-21munin-run09:45:000
3281800994636,6cyclictest3468939-21head12:21:012
3281798994639,4cyclictest3435217-21systemctl11:40:000
3281800994537,3cyclictest0-21swapper/211:50:012
3281800994435,6cyclictest421-21dbus-daemon09:05:012
3281800994435,6cyclictest421-21dbus-daemon09:05:012
3281798994335,5cyclictest3401188-21unixbench-2d10:56:050
3281798994332,8cyclictest3341930-21munin-run09:40:010
3281798994332,8cyclictest3341930-21munin-run09:40:010
3281800994233,6cyclictest421-21dbus-daemon09:55:012
3281799994233,6cyclictest421-21dbus-daemon11:15:011
328179899425,31cyclictest3323259-21ssh09:15:550
3281798994236,3cyclictest0-21swapper/009:25:010
3281798994233,6cyclictest3284822-21systemd-run07:20:020
3281798994232,7cyclictest3399328-21munin-plugin-st10:55:010
3281798994231,7cyclictest3395423-21systemd-run10:50:010
3281798994227,12cyclictest3377844-21grep10:26:020
3281798994214,22cyclictest3428616-21ssh11:30:540
3281800994133,5cyclictest3333995-21idleruntime-cro09:30:012
3281800994131,6cyclictest1-21systemd09:20:012
3281799994133,5cyclictest3395423-21systemd-run10:50:011
3281799994133,5cyclictest3372440-21grep10:20:001
3281799994123,12cyclictest271ktimers/108:40:521
3281799994123,12cyclictest271ktimers/108:40:521
328179899414,34cyclictest3331279-21ssh09:25:510
3281798994126,12cyclictest3448366-21ntp_kernel_err11:55:490
3281800994033,4cyclictest3407279-21cron11:05:012
3281800994033,4cyclictest3407279-21cron11:05:012
3281800994032,5cyclictest3341932-21munin-plugin-st09:40:012
3281800994032,5cyclictest3341932-21munin-plugin-st09:40:012
3281800994030,7cyclictest3361592-21cron10:05:012
3281799994031,4cyclictest3292261-21kworker/u8:2-rpciod07:59:571
328179899402,7cyclictest14-21ksoftirqd/008:50:530
328179899402,26cyclictest3412721-21ssh11:10:560
3281800993932,4cyclictest3399365-21latency_hist10:55:022
3281800993928,7cyclictest3474977-21cron12:30:012
328179999394,9cyclictest171rcu_preempt09:00:511
328179999394,9cyclictest171rcu_preempt09:00:511
3281799993931,5cyclictest3467004-21munin-plugin-st12:20:001
3281798993929,7cyclictest3311797-21systemd08:50:010
3281798993912,21cyclictest3351106-21ssh09:50:470
3281800993830,4cyclictest0-21swapper/210:40:532
3281800993827,8cyclictest3346921-21kernelversion09:45:452
3281799993830,5cyclictest3400204-21fschecks_count10:55:461
3281799993828,6cyclictest0-21swapper/109:50:471
3281799993813,19cyclictest1051-21mta-sts-daemon09:05:521
328179899389,22cyclictest3416667-21ssh11:15:550
3281798993831,4cyclictest3447143-21munin-plugin-st11:55:010
328179899381,28cyclictest2469968-21cups-browsed11:00:520
328179899381,28cyclictest2469968-21cups-browsed11:00:520
3281800993730,3cyclictest0-21swapper/212:35:532
3281800993729,5cyclictest3372431-21ssh10:20:002
3281800993728,5cyclictest3295288-21kworker/u8:907:55:012
3281798993726,8cyclictest151ktimers/008:55:500
328179899372,30cyclictest3420517-21ssh11:20:480
3281798993722,4cyclictest0-21swapper/012:15:390
328179899371,28cyclictest3362845-21ssh10:05:500
3281800993629,4cyclictest3345924-21kworker/u8:15-xprtiod10:35:012
3281800993628,5cyclictest3357769-21awk10:00:012
3281800993627,6cyclictest3421133-21unixbench_multi11:21:002
3281799993630,3cyclictest0-21swapper/111:25:441
3281799993630,3cyclictest0-21swapper/110:45:011
3281799993628,5cyclictest3482958-21idleruntime-cro12:40:011
3281799993628,5cyclictest3478979-21/usr/sbin/munin12:35:011
3281799993628,5cyclictest3341968-21systemd-run09:40:011
3281799993628,5cyclictest3341968-21systemd-run09:40:011
3281799993627,7cyclictest3347648-21timerandwakeup09:46:011
328179899363,29cyclictest3472320-21ssh12:25:510
3281798993627,6cyclictest0-21swapper/010:21:030
3281798993627,3cyclictest171rcu_preempt10:10:570
3281798993623,9cyclictest3456423-21rm12:05:540
3281798993618,11cyclictest12350irq/32-eth009:30:520
3281800993528,4cyclictest3321835-21munin-plugin-st09:15:002
3281800993525,7cyclictest3366596-21sshd10:10:542
3281800993524,7cyclictest2469968-21cups-browsed12:05:592
3281799993528,4cyclictest0-21swapper/108:20:011
3281799993526,7cyclictest3448510-21sh11:55:541
3281799993526,6cyclictest420-21avahi-daemon11:43:261
3281799993525,7cyclictest3396926-21mailstats10:51:001
328179999352,24cyclictest3440566-21ssh11:45:531
3281798993528,4cyclictest3349894-21systemd-run09:50:010
3281798993528,4cyclictest12450irq/33-eth012:20:530
3281798993523,9cyclictest3424792-21mailstats11:25:580
3281798993522,9cyclictest3317777-21diskmemload12:10:500
3281798993522,9cyclictest3317777-21diskmemload12:10:500
3281800993428,3cyclictest0-21swapper/211:40:522
3281800993422,5cyclictest1056-21mta-sts-daemon08:35:432
3281800993422,5cyclictest1056-21mta-sts-daemon08:35:432
3281799993421,10cyclictest3359338-21mailstats10:01:001
3281799993411,9cyclictest171rcu_preempt09:14:251
328179899345,15cyclictest3480250-21ssh12:35:500
3281798993427,3cyclictest0-21swapper/011:05:450
3281798993424,7cyclictest3387605-21latency_hist10:40:020
328179899342,20cyclictest3355224-21ntpq09:55:540
3281798993412,17cyclictest191rcuc/008:05:510
328180099336,7cyclictest171rcu_preempt08:50:472
3281800993327,3cyclictest0-21swapper/211:10:502
3281800993326,4cyclictest0-21swapper/212:35:012
3281800993326,4cyclictest0-21swapper/209:10:002
3281800993325,5cyclictest3393379-21sed10:46:022
3281800993324,6cyclictest3289266-21cron07:35:012
3281800993323,7cyclictest3335254-21ntp_kernel_pll_09:30:522
3281799993326,4cyclictest3335824-21grep09:31:031
3281799993326,4cyclictest3298294-21munin-plugin-st08:05:011
3281799993325,5cyclictest3460886-21tail12:11:011
3281799993325,5cyclictest3460886-21tail12:11:011
3281799993325,5cyclictest3342780-21diskmemload09:40:421
3281799993325,5cyclictest2469968-21cups-browsed12:25:251
3281799993320,10cyclictest3373442-21latency10:20:461
328179899339,2cyclictest171rcu_preempt07:10:530
3281798993326,3cyclictest0-21swapper/012:05:010
3281798993316,10cyclictest3388906-21ssh10:40:540
328179899331,5cyclictest3305160-21ntpq08:25:500
328180099323,22cyclictest3464316-21ssh12:15:512
3281800993226,3cyclictest0-21swapper/212:15:012
3281800993226,3cyclictest0-21swapper/212:15:012
3281800993226,3cyclictest0-21swapper/210:40:022
3281800993225,4cyclictest3376058-21cron10:25:002
3281800993223,6cyclictest3365395-21idleruntime-cro10:10:012
328180099322,23cyclictest3408648-21ssh11:05:532
328180099322,23cyclictest3306739-21sshd08:30:532
3281799993226,3cyclictest0-21swapper/111:20:011
3281799993224,5cyclictest3405255-21sed11:01:021
3281799993224,5cyclictest3405255-21sed11:01:021
3281799993224,5cyclictest3362452-21fschecks_count10:05:441
3281799993224,5cyclictest3357751-21cron10:00:011
3281799993224,5cyclictest3293680-21unixbench_multi07:46:001
3281799993223,6cyclictest439-21systemd-logind07:45:011
3281799993223,6cyclictest261-21systemd-journal12:24:541
3281799993223,5cyclictest0-21swapper/111:50:551
3281799993221,8cyclictest620-21sshd12:01:541
3281798993225,5cyclictest3290803-21kworker/u8:0-nfsiod07:59:570
3281798993225,4cyclictest3441057-21ssh11:46:020
3281798993219,3cyclictest171rcu_preempt10:00:510
3281798993219,10cyclictest12350irq/32-eth009:10:510
3281800993126,2cyclictest0-21swapper/208:00:012
3281800993125,4cyclictest1056-21mta-sts-daemon11:26:472
3281800993125,3cyclictest0-21swapper/212:00:552
3281800993125,3cyclictest0-21swapper/208:15:012
3281800993125,3cyclictest0-21swapper/207:15:012
3281800993124,4cyclictest3327490-21ssh09:21:002
3281800993122,6cyclictest3401217-21sed10:56:052
3281800993121,4cyclictest0-21swapper/211:15:492
328180099310,8cyclictest171rcu_preempt08:45:562
3281799993125,3cyclictest0-21swapper/112:05:591
3281799993125,3cyclictest0-21swapper/111:05:531
3281799993124,4cyclictest3314764-21munin-run09:00:011
3281799993124,4cyclictest1054-21mta-sts-daemon09:20:531
3281799993123,5cyclictest3283166-21tail07:11:001
3281799993122,6cyclictest3400680-21kworker/u8:7+nfsiod11:35:511
3281799993119,9cyclictest3420634-21ssh11:20:531
3281798993123,5cyclictest3308648-21tail08:36:020
3281798993123,5cyclictest3308648-21tail08:36:020
3281798993123,5cyclictest3307290-21systemd-run08:35:010
3281798993117,3cyclictest171rcu_preempt08:40:550
3281798993117,3cyclictest171rcu_preempt08:40:550
3281798993113,4cyclictest171rcu_preempt07:20:520
328180099303,20cyclictest3317777-21diskmemload09:40:522
3281800993025,2cyclictest0-21swapper/207:45:432
3281800993022,5cyclictest2469968-21cups-browsed08:25:562
3281800993021,6cyclictest0-21swapper/208:15:532
3281799993022,5cyclictest695-21cron08:40:001
3281799993022,5cyclictest695-21cron08:40:001
3281799993022,5cyclictest3313257-21cron08:55:011
3281798993024,3cyclictest0-21swapper/007:35:010
3281798993022,5cyclictest421-21dbus-daemon11:40:410
3281798993022,5cyclictest3475412-21sh12:30:260
3281798993021,6cyclictest3301157-21tail08:11:030
3281798993017,7cyclictest171rcu_preempt07:40:550
3281800992923,3cyclictest3372486-21kworker/u8:9+xprtiod12:00:012
3281800992919,7cyclictest1051-21mta-sts-daemon08:55:492
3281799992924,4cyclictest3281797-21cyclictest10:30:501
328179999292,21cyclictest547-21gmain10:37:591
3281799992920,6cyclictest3295273-21grep07:55:011
3281799992917,5cyclictest3304284-21latency_hist08:25:001
328179899294,21cyclictest3302194-21ntpq08:15:530
3281800992821,4cyclictest1054-21mta-sts-daemon07:22:022
3281800992819,6cyclictest3444771-21systemctl11:51:032
328179999285,10cyclictest171rcu_preempt09:25:511
328179999283,21cyclictest3311226-21sshd08:45:541
3281799992822,4cyclictest3369011-21systemd10:15:021
328179999282,21cyclictest3299179-21sshd08:05:531
3281799992820,5cyclictest3306298-21fschecks_count08:30:431
3281799992820,5cyclictest0-21swapper/107:35:531
3281798992824,3cyclictest151ktimers/007:35:520
328179899281,22cyclictest3287240-21sshd07:25:530
328180099271,22cyclictest3377321-21ssh10:25:522
328179999272,20cyclictest3323116-21ssh09:15:491
3281799992721,3cyclictest3286277-21kworker/u8:4+rpciod07:25:021
3281799992720,4cyclictest1054-21mta-sts-daemon10:25:161
3281799992718,7cyclictest3427689-21sh11:30:301
328179899272,7cyclictest0-21swapper/008:20:510
3281798992722,2cyclictest0-21swapper/009:10:010
328180099262,18cyclictest3299182-21sshd08:05:532
3281800992621,2cyclictest0-21swapper/208:05:012
3281800992615,7cyclictest3304283-21cron08:25:002
328179999269,11cyclictest261rcuc/108:10:551
3281799992619,4cyclictest2821093-21kworker/u8:10-xprtiod07:25:521
3281798992616,3cyclictest171rcu_preempt07:46:010
3281798992611,12cyclictest0-21swapper/008:05:010
3281800992515,7cyclictest659-21snmpd07:15:572
328179899251,20cyclictest2469968-21cups-browsed07:50:520
3281800992419,2cyclictest0-21swapper/208:45:002
3281800992419,2cyclictest0-21swapper/208:45:002
3281800992417,4cyclictest2469968-21cups-browsed07:40:352
3281800992317,3cyclictest1055-21mta-sts-daemon07:25:552
328179999234,4cyclictest3288301-21fschecks_count07:30:421
3281799992315,4cyclictest0-21swapper/107:15:481
3281799992313,7cyclictest421-21dbus-daemon07:10:021
3281798992311,6cyclictest171rcu_preempt09:00:550
3281798992311,6cyclictest171rcu_preempt09:00:550
328180199223,10cyclictest0-21swapper/308:30:543
328180099221,16cyclictest659-21snmpd07:35:532
3281799992216,3cyclictest0-21swapper/108:25:531
328180199214,10cyclictest0-21swapper/310:50:553
328180199212,9cyclictest0-21swapper/312:15:493
328180199212,8cyclictest0-21swapper/310:45:433
328180199203,11cyclictest0-21swapper/310:35:543
328180199202,9cyclictest0-21swapper/309:40:493
328180199202,11cyclictest0-21swapper/311:35:543
328180199202,11cyclictest0-21swapper/311:10:513
328180199202,11cyclictest0-21swapper/310:05:513
328180199202,10cyclictest0-21swapper/311:25:533
328180199202,10cyclictest0-21swapper/309:20:503
3281800992011,5cyclictest261-21systemd-journal07:10:022
3281798992012,3cyclictest171rcu_preempt07:10:020
328180199193,9cyclictest0-21swapper/312:00:453
328180199193,9cyclictest0-21swapper/310:30:533
328180199193,9cyclictest0-21swapper/309:50:473
328180199193,9cyclictest0-21swapper/309:45:453
328180199193,8cyclictest0-21swapper/308:35:513
328180199193,8cyclictest0-21swapper/308:35:513
328180199192,7cyclictest0-21swapper/308:55:543
328180199192,11cyclictest0-21swapper/311:15:503
328180199192,10cyclictest0-21swapper/312:20:553
328180199192,10cyclictest0-21swapper/312:00:013
328180199192,10cyclictest0-21swapper/311:00:483
328180199192,10cyclictest0-21swapper/311:00:483
328180199192,10cyclictest0-21swapper/309:25:523
328180199186,7cyclictest0-21swapper/311:05:493
328180199184,8cyclictest0-21swapper/311:50:563
328180199183,8cyclictest0-21swapper/312:05:593
328180199183,6cyclictest0-21swapper/308:10:533
328180199183,10cyclictest0-21swapper/309:30:553
328180199182,9cyclictest0-21swapper/312:25:503
328180199182,10cyclictest0-21swapper/310:40:553
328180199182,10cyclictest0-21swapper/309:55:513
328180199181,11cyclictest0-21swapper/310:25:553
328180199176,5cyclictest0-21swapper/309:15:513
328180199174,7cyclictest0-21swapper/308:40:513
328180199174,7cyclictest0-21swapper/308:40:513
328180199173,9cyclictest0-21swapper/312:31:013
328180199173,9cyclictest0-21swapper/307:59:563
328180199173,8cyclictest0-21swapper/311:20:423
328180199173,8cyclictest0-21swapper/310:55:543
328180199172,9cyclictest0-21swapper/311:45:503
328180199172,7cyclictest0-21swapper/308:45:543
328180199172,10cyclictest0-21swapper/310:16:013
328180199171,9cyclictest0-21swapper/309:35:423
328180199171,9cyclictest0-21swapper/309:35:423
328180199171,10cyclictest0-21swapper/309:10:463
328180199171,10cyclictest0-21swapper/307:50:543
328180199164,7cyclictest0-21swapper/307:20:013
328180199164,5cyclictest0-21swapper/308:25:503
328180199164,4cyclictest0-21swapper/307:40:483
328180199162,9cyclictest0-21swapper/312:10:443
328180199162,9cyclictest0-21swapper/312:10:443
328180199162,8cyclictest0-21swapper/310:20:503
328180199162,7cyclictest0-21swapper/307:35:523
328180199161,9cyclictest0-21swapper/311:30:473
328180199161,9cyclictest0-21swapper/310:10:483
328180199161,9cyclictest0-21swapper/310:00:403
328180199161,7cyclictest0-21swapper/308:20:543
328180199161,6cyclictest0-21swapper/309:00:503
328180199161,6cyclictest0-21swapper/309:00:503
328180199152,9cyclictest0-21swapper/312:35:503
328180199152,8cyclictest0-21swapper/311:40:413
328180199152,8cyclictest0-21swapper/307:35:013
328180199152,7cyclictest0-21swapper/307:25:573
328180199143,6cyclictest0-21swapper/308:05:533
328180199142,8cyclictest0-21swapper/309:10:013
328180199142,7cyclictest0-21swapper/307:20:533
328180199142,7cyclictest0-21swapper/307:10:483
328180199142,6cyclictest0-21swapper/308:15:493
328180199134,6cyclictest0-21swapper/307:10:023
328180199133,5cyclictest0-21swapper/308:00:483
328180199131,8cyclictest0-21swapper/308:50:473
328180199131,7cyclictest0-21swapper/307:45:523
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional