You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-15 - 17:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Wed Apr 15, 2026 12:43:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
268589995340,9cyclictest3639778-21cups-browsed11:48:211
26858899537,8cyclictest427370-21ssh11:40:420
268588995321,24cyclictest12350irq/32-eth009:30:370
268589995244,6cyclictest271ktimers/109:55:411
268588994938,6cyclictest151ktimers/008:55:390
268588994937,9cyclictest333036-21sh09:45:010
26858899491,22cyclictest443414-21ssh12:00:400
268588994836,9cyclictest341087-21/usr/sbin/munin09:55:010
268590994637,6cyclictest1-21systemd12:00:012
268589994636,6cyclictest0-21swapper/109:20:011
268588994634,5cyclictest467857-21grep12:30:480
268588994629,4cyclictest268769-21latency_hist07:10:030
268589994534,7cyclictest1-21systemd09:45:021
268588994530,9cyclictest12450irq/33-eth010:00:390
26858899452,17cyclictest358440-21ntpq10:15:340
26858899452,14cyclictest269502-21ntpq07:10:390
268588994512,30cyclictest421-21dbus-daemon10:42:260
268590994436,5cyclictest349225-21idleruntime-cro10:05:012
268588994438,3cyclictest0-21swapper/011:35:300
268588994436,5cyclictest277616-21systemd-run07:40:010
268588994434,7cyclictest432921-21sshd11:47:430
268588994432,9cyclictest12350irq/32-eth009:45:460
268588994431,5cyclictest316850-21ssh09:25:020
268588994431,5cyclictest316850-21ssh09:25:020
268588994426,15cyclictest3639778-21cups-browsed10:11:390
268588994426,15cyclictest3639778-21cups-browsed10:11:390
268588994415,26cyclictest3639778-21cups-browsed12:09:590
26858899441,12cyclictest278478-21ntpq07:40:360
268590994335,5cyclictest421-21dbus-daemon07:25:022
268590994333,6cyclictest420-21avahi-daemon11:42:062
268590994333,3cyclictest0-21swapper/209:15:412
26858899433,7cyclictest271013-21ntpq07:15:380
268588994326,14cyclictest3639778-21cups-browsed10:38:130
268588994318,22cyclictest548-21gdbus07:50:450
26858899431,14cyclictest407086-21sh11:15:410
268588994310,17cyclictest411111-21ssh11:20:380
26858899430,11cyclictest302514-21ntpq09:00:380
26859099424,30cyclictest415269-21ssh11:25:422
26858899428,19cyclictest370629-21ssh10:30:360
268588994224,14cyclictest0-21swapper/011:05:450
268588994219,19cyclictest0-21swapper/009:40:020
26858899421,8cyclictest295476-21/usr/sbin/munin08:35:460
268588994217,22cyclictest420-21avahi-daemon10:00:030
268588994211,2cyclictest0-21swapper/008:25:360
26858899421,11cyclictest395032-21ntpq11:00:390
268590994134,4cyclictest446158-21ssh12:05:012
268590994133,5cyclictest421-21dbus-daemon12:25:012
268590994130,7cyclictest261-21systemd-journal11:22:422
26858999414,7cyclictest171rcu_preempt07:10:421
268589994135,3cyclictest0-21swapper/111:50:111
268589994133,5cyclictest405895-21systemd-run11:15:011
268589994131,7cyclictest3639778-21cups-browsed08:45:011
268589994126,11cyclictest271ktimers/107:45:361
268588994134,3cyclictest0-21swapper/011:30:350
268588994132,6cyclictest387586-21vmstat10:50:490
268588994132,6cyclictest387586-21vmstat10:50:490
268588994114,24cyclictest0-21swapper/008:20:420
26858899411,3cyclictest309883-21ntpq09:15:380
26858899411,17cyclictest289130-21sendmail_mailqu08:15:430
26858899411,17cyclictest289130-21sendmail_mailqu08:15:430
26858899411,14cyclictest287499-21ntpq08:10:380
26858899411,14cyclictest287499-21ntpq08:10:380
26858899411,11cyclictest402073-21ssh11:10:030
26858899411,11cyclictest362676-21ntpq10:20:400
268590994030,7cyclictest402549-21ssh11:10:302
268590994030,5cyclictest0-21swapper/209:50:432
268590994029,7cyclictest0-21swapper/209:25:012
268590994029,7cyclictest0-21swapper/209:25:012
268589994032,5cyclictest1055-21mta-sts-daemon08:45:301
268589994030,7cyclictest421-21dbus-daemon09:36:341
268589994023,9cyclictest373538-21cron10:35:011
268588994033,4cyclictest454157-21systemd-run12:15:020
268588994031,6cyclictest420-21avahi-daemon12:35:070
268588994015,22cyclictest318025-21ntp_kernel_pll_09:25:370
268588994013,3cyclictest0-21swapper/010:10:010
26858899401,12cyclictest415158-21ntpq11:25:380
26858899401,11cyclictest272549-21ntpq07:20:390
268590993932,4cyclictest361355-21cron10:20:002
268590993930,6cyclictest308686-21systemd-run09:15:012
268589993932,3cyclictest0-21swapper/111:05:441
268589993918,14cyclictest271ktimers/109:50:411
26858899393,6cyclictest303952-21ntpq09:05:360
268588993932,5cyclictest391562-21sed10:55:490
268588993931,4cyclictest0-21swapper/012:20:400
26858899392,7cyclictest293415-21ntpq08:30:350
26858899392,13cyclictest421-21dbus-daemon12:19:090
268588993920,16cyclictest3639778-21cups-browsed11:55:300
268588993919,16cyclictest0-21swapper/010:25:360
268588993916,19cyclictest0-21swapper/010:45:380
268588993916,19cyclictest0-21swapper/010:45:380
26858899390,14cyclictest286009-21ntpq08:05:390
268590993830,5cyclictest365428-21cron10:25:002
268590993829,7cyclictest401856-21sh11:10:012
268589993830,4cyclictest0-21swapper/111:20:011
268589993829,6cyclictest467764-21unixbench-2d12:30:471
268589993829,6cyclictest1055-21mta-sts-daemon12:05:571
268589993827,8cyclictest313911-21ntp_kernel_pll_09:20:361
268589993827,8cyclictest313911-21ntp_kernel_pll_09:20:361
268588993826,9cyclictest282599-21fschecks_time07:55:320
268588993817,19cyclictest0-21swapper/011:50:300
268588993813,22cyclictest465360-21sh12:28:410
268588993812,2cyclictest0-21swapper/009:10:300
268588993812,2cyclictest0-21swapper/008:40:420
26859099373,26cyclictest378757-21ssh10:40:352
268590993732,2cyclictest0-21swapper/212:40:012
26858999376,4cyclictest171rcu_preempt10:25:351
268589993728,6cyclictest3639778-21cups-browsed10:15:301
268589993719,10cyclictest276303-21/usr/sbin/munin07:35:281
26858899370,8cyclictest280514-21unixbench_singl07:45:490
268590993628,5cyclictest325888-21kernelversion09:35:362
268590993627,5cyclictest321436-21aten2.4-expect09:30:262
268590993626,7cyclictest297079-21/usr/sbin/munin08:45:022
268589993631,2cyclictest0-21swapper/111:00:001
268589993627,6cyclictest415765-21unixbench_singl11:25:481
268589993625,8cyclictest474124-21latency_hist12:40:021
26858899361,17cyclictest274848-21aten_r2power_cu07:30:240
26858899361,11cyclictest274589-21/usr/sbin/munin07:30:020
268590993527,5cyclictest370039-21head10:30:272
268590993525,8cyclictest418471-21aten_r2power_cu11:30:262
268590993523,8cyclictest447181-21missed_timers12:05:332
268589993529,3cyclictest0-21swapper/112:25:011
268589993529,3cyclictest0-21swapper/112:05:011
268589993527,5cyclictest375313-21unixbench-2d10:35:491
268589993525,8cyclictest516-21in:imuxsock07:25:011
26858899352,7cyclictest284508-21ntpq08:00:390
268590993426,5cyclictest383492-21unixbench_multi10:45:502
268590993426,5cyclictest383492-21unixbench_multi10:45:502
268590993426,5cyclictest261-21systemd-journal12:35:002
26859099342,5cyclictest171rcu_preempt12:25:422
268590993425,6cyclictest1-21systemd10:55:012
268590993425,6cyclictest1-21systemd10:55:012
268590993425,6cyclictest1-21systemd08:48:422
268590993425,5cyclictest391529-21/usr/sbin/munin10:55:492
268589993427,4cyclictest0-21swapper/109:15:011
268589993423,8cyclictest346472-21sshd10:00:421
268589993423,7cyclictest548-21gdbus12:12:331
26858999342,26cyclictest350438-21ssh10:05:371
268590993327,3cyclictest0-21swapper/209:45:012
268590993327,3cyclictest0-21swapper/208:25:012
268590993327,3cyclictest0-21swapper/208:20:002
268590993327,3cyclictest0-21swapper/208:20:002
268590993326,4cyclictest345181-21systemd-run10:00:012
268590993326,4cyclictest0-21swapper/210:05:342
268590993325,5cyclictest293098-21fschecks_time08:30:302
268590993323,4cyclictest0-21swapper/211:45:382
26858999334,23cyclictest395038-21ssh11:00:391
268589993328,3cyclictest304602-21kworker/u8:4-rpciod10:25:011
268589993328,2cyclictest0-21swapper/110:50:431
268589993328,2cyclictest0-21swapper/110:50:431
268589993327,3cyclictest0-21swapper/111:35:011
268589993326,4cyclictest1054-21mta-sts-daemon11:37:551
268589993324,7cyclictest301605-21latency_hist09:00:021
268589993324,6cyclictest439-21systemd-logind09:35:011
268589993322,8cyclictest379077-21mailstats10:40:441
26858899332,6cyclictest299409-21ntpq08:50:330
26858899332,6cyclictest298319-21wc08:45:460
26859099324,18cyclictest451303-21ssh12:10:352
268590993224,6cyclictest365974-21aten_r2power_fr10:25:262
268590993222,8cyclictest334148-21collect209:45:382
26859099322,24cyclictest620-21sshd09:00:422
268590993221,8cyclictest406287-21/usr/sbin/munin11:15:352
268589993226,3cyclictest0-21swapper/111:21:431
268589993225,4cyclictest274585-21sh07:30:011
268589993224,5cyclictest385742-21systemd-run10:50:011
268589993224,5cyclictest385742-21systemd-run10:50:011
268589993224,5cyclictest304611-21latency_hist09:10:021
268589993224,5cyclictest0-21swapper/108:55:021
268590993124,4cyclictest377607-21cron10:40:012
268590993123,5cyclictest3639778-21cups-browsed07:44:072
268589993125,3cyclictest0-21swapper/108:20:001
268589993125,3cyclictest0-21swapper/108:20:001
268589993124,3cyclictest0-21swapper/107:40:341
268589993123,5cyclictest442134-21idleruntime-cro12:00:011
268589993123,5cyclictest334790-21unixbench_multi09:45:531
26859199303,9cyclictest0-21swapper/310:20:353
26859099303,21cyclictest428-21mta-sts-daemon07:45:382
268590993024,4cyclictest268587-21cyclictest12:20:012
268590993023,4cyclictest357321-21grep10:15:012
268590993023,4cyclictest357321-21grep10:15:012
268590993023,4cyclictest270090-21systemd-run07:15:012
268590993023,3cyclictest0-21swapper/211:00:132
268590993021,7cyclictest423341-21ssh11:35:422
268590993020,7cyclictest3639759-21cupsd09:26:392
268589993022,6cyclictest318225-21processes09:25:431
268589993022,5cyclictest426837-21fschecks_count11:40:311
268589993021,6cyclictest1051-21mta-sts-daemon07:30:381
268589993021,5cyclictest463832-21grep12:25:491
26859199293,8cyclictest0-21swapper/309:50:423
268590992922,4cyclictest438142-21systemd-run11:55:012
268590992922,4cyclictest286458-21unixbench_multi08:05:482
268590992921,5cyclictest295442-21unixbench_multi08:35:462
268590992921,5cyclictest288085-21systemd-run08:15:012
268590992921,5cyclictest288085-21systemd-run08:15:012
268590992921,5cyclictest283554-21cron08:00:002
268590992920,4cyclictest0-21swapper/208:00:392
26858999292,21cyclictest354453-21ssh10:10:361
26858999292,21cyclictest354453-21ssh10:10:361
268589992921,5cyclictest454889-21fschecks_time12:15:301
26859099283,19cyclictest277027-21sendmail-mta07:35:382
268590992821,4cyclictest291069-21kworker/u8:4+xprtiod09:00:012
268589992821,5cyclictest1055-21mta-sts-daemon07:50:421
26859199273,10cyclictest0-21swapper/310:25:363
26859199272,7cyclictest0-21swapper/311:05:393
268590992719,5cyclictest421-21dbus-daemon07:20:022
268590992716,8cyclictest275451-21ntp_kernel_pll_07:30:342
268590992714,3cyclictest171rcu_preempt08:30:012
268589992722,2cyclictest0-21swapper/107:20:011
268589992721,3cyclictest0-21swapper/108:10:001
268589992720,4cyclictest1056-21mta-sts-daemon08:30:361
268589992719,5cyclictest303079-21munin-plugin-st09:05:011
268590992617,7cyclictest303605-21uniq09:05:302
26859199253,6cyclictest0-21swapper/311:50:383
26859099252,16cyclictest281509-21sshd07:50:422
268590992511,3cyclictest171rcu_preempt07:25:302
268589992518,4cyclictest0-21swapper/108:00:011
268589992516,6cyclictest421-21dbus-daemon07:10:021
268590992416,5cyclictest268613-21latency_hist07:10:022
26858999245,11cyclictest271ktimers/108:10:351
26858999245,11cyclictest271ktimers/108:10:351
26858999242,4cyclictest171rcu_preempt08:00:391
26858999242,17cyclictest290608-21ps08:20:431
268589992419,2cyclictest0-21swapper/108:25:461
268589992315,5cyclictest295546-21kworker/u8:11+xprtiod08:40:011
26859099222,14cyclictest299426-21ntpq08:50:342
26859199202,8cyclictest0-21swapper/312:00:363
26859199202,10cyclictest0-21swapper/309:35:413
26859199193,9cyclictest0-21swapper/309:45:443
26859199193,9cyclictest0-21swapper/309:25:413
26859199193,9cyclictest0-21swapper/309:10:353
26859199192,8cyclictest0-21swapper/310:30:353
26859199184,8cyclictest0-21swapper/311:40:383
26859199184,7cyclictest0-21swapper/311:55:403
26859199184,7cyclictest0-21swapper/310:15:363
26859199183,9cyclictest0-21swapper/309:55:373
26859199183,7cyclictest0-21swapper/312:25:393
26859199182,9cyclictest0-21swapper/312:20:313
26859199182,8cyclictest0-21swapper/309:40:353
26859199182,10cyclictest0-21swapper/310:55:343
26859199181,9cyclictest0-21swapper/311:45:393
26859199174,7cyclictest0-21swapper/311:00:373
26859199173,9cyclictest0-21swapper/309:30:343
26859199173,8cyclictest0-21swapper/312:10:373
26859199173,8cyclictest0-21swapper/308:15:413
26859199173,8cyclictest0-21swapper/308:15:413
26859199172,9cyclictest0-21swapper/311:30:363
26859199171,9cyclictest0-21swapper/311:20:313
26859199171,9cyclictest0-21swapper/311:15:383
26859199171,9cyclictest0-21swapper/310:00:343
26859199171,9cyclictest0-21swapper/307:35:403
26859199164,8cyclictest0-21swapper/310:10:473
26859199164,8cyclictest0-21swapper/310:10:473
26859199164,6cyclictest0-21swapper/309:20:343
26859199164,6cyclictest0-21swapper/309:20:343
26859199163,7cyclictest0-21swapper/310:50:363
26859199163,7cyclictest0-21swapper/310:50:363
26859199162,9cyclictest0-21swapper/312:05:253
26859199162,9cyclictest0-21swapper/311:25:393
26859199162,8cyclictest0-21swapper/310:45:383
26859199162,8cyclictest0-21swapper/310:45:383
26859199162,8cyclictest0-21swapper/309:15:313
26859199162,7cyclictest0-21swapper/307:55:423
26859199161,7cyclictest0-21swapper/307:10:363
26859199161,6cyclictest0-21swapper/309:00:353
26859199154,5cyclictest0-21swapper/307:45:403
26859199153,7cyclictest0-21swapper/312:30:343
26859199153,6cyclictest0-21swapper/310:40:333
26859199152,8cyclictest0-21swapper/312:15:363
26859199152,8cyclictest0-21swapper/310:35:383
26859199152,8cyclictest0-21swapper/308:10:013
26859199152,7cyclictest0-21swapper/307:20:463
26859199152,6cyclictest0-21swapper/308:40:373
26859199151,9cyclictest0-21swapper/308:10:423
26859199151,9cyclictest0-21swapper/308:10:423
26859199151,8cyclictest0-21swapper/311:10:373
26859199151,8cyclictest0-21swapper/310:05:253
26859199151,7cyclictest0-21swapper/309:05:413
26859199151,7cyclictest0-21swapper/308:50:373
26859199145,6cyclictest0-21swapper/312:35:403
26859199143,7cyclictest0-21swapper/311:35:323
26859199142,7cyclictest0-21swapper/307:35:023
26859199142,6cyclictest0-21swapper/308:30:363
26859199142,6cyclictest0-21swapper/308:20:373
26859199141,7cyclictest0-21swapper/308:25:413
26859199141,6cyclictest0-21swapper/307:15:343
26859199132,6cyclictest0-21swapper/307:40:363
26859199132,5cyclictest0-21swapper/307:50:383
26859199132,5cyclictest0-21swapper/307:10:033
26859199131,7cyclictest0-21swapper/308:45:373
26859199121,7cyclictest0-21swapper/308:00:353
26859199121,6cyclictest0-21swapper/308:55:303
26859199113,6cyclictest0-21swapper/308:35:283
26859199111,6cyclictest0-21swapper/307:25:373
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional