You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 14:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Fri Apr 17, 2026 12:43:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4161115996150,8cyclictest95754-21systemd-run11:05:010
4161116996048,6cyclictest0-21swapper/110:30:381
4161117995645,7cyclictest421-21dbus-daemon11:00:002
4161116995546,6cyclictest32560-21ssh09:45:371
4161117995328,21cyclictest0-21swapper/212:40:002
4161117995328,21cyclictest0-21swapper/212:40:002
4161115995241,5cyclictest0-21swapper/011:25:430
416111699492,40cyclictest137626-21ssh12:00:371
4161117994843,2cyclictest0-21swapper/210:35:022
4161116994837,6cyclictest0-21swapper/112:20:341
416111799471,16cyclictest4187565-21ntpq08:35:432
4161116994739,3cyclictest0-21swapper/110:15:361
416111599472,39cyclictest81064-21ssh10:45:380
416111599472,39cyclictest81064-21ssh10:45:380
4161115994723,17cyclictest60982-21diskmemload10:20:390
4161117994635,7cyclictest85680-21unixbench_singl10:50:502
416111799462,17cyclictest161429-21ps12:30:452
4161115994639,5cyclictest261-21systemd-journal11:50:000
4161115994639,5cyclictest261-21systemd-journal11:50:000
4161115994636,7cyclictest35477-21sh09:50:010
4161115994634,5cyclictest12464-21modprobe09:20:380
4161117994526,15cyclictest4164342-21/usr/sbin/munin07:20:312
4161116994536,6cyclictest421-21dbus-daemon10:25:291
4161115994527,15cyclictest0-21swapper/009:35:370
4161115994436,6cyclictest12350irq/32-eth009:15:380
4161115994434,6cyclictest12350irq/32-eth010:05:400
416111599442,31cyclictest52943-21ssh10:10:410
4161117994334,6cyclictest4180614-21systemd-run08:15:012
4161117994331,7cyclictest3345305-21cups-browsed09:25:162
4161117994326,13cyclictest0-21swapper/210:00:372
4161117994317,23cyclictest11308-21latency_hist09:20:012
416111599433,28cyclictest145617-21ssh12:10:450
416111799422,6cyclictest92973-21ntpq11:00:392
416111799422,6cyclictest4177089-21ntpq08:00:422
416111799421,7cyclictest428-21mta-sts-daemon12:23:162
416111799421,10cyclictest54866-21ssh10:13:262
416111799421,10cyclictest118422-21ssh11:35:082
416111799421,10cyclictest111223-21ssh11:25:042
4161116994234,5cyclictest27336-21cut09:40:011
4161116994234,3cyclictest0-21swapper/111:10:371
416111699423,32cyclictest77124-21ssh10:40:391
4161116994228,10cyclictest36400-21latency09:50:321
416111799414,12cyclictest428-21mta-sts-daemon12:11:012
4161117994122,16cyclictest3345305-21cups-browsed09:49:392
416111799411,9cyclictest60086-21ssh10:20:152
416111799411,9cyclictest135793-21ssh11:58:452
416111799411,7cyclictest4189033-21ntpq08:40:392
416111799411,7cyclictest4189033-21ntpq08:40:392
416111799411,6cyclictest149296-21ntpq12:15:352
416111799411,11cyclictest43142-21ssh09:59:162
416111799411,11cyclictest43142-21ssh09:59:162
416111799411,10cyclictest27754-21ssh09:40:212
416111799411,10cyclictest128087-21ssh11:48:202
416111799411,10cyclictest128087-21ssh11:48:202
416111799411,10cyclictest106882-21ssh11:19:092
416111799410,9cyclictest59684-21ssh10:19:572
4161116994135,3cyclictest0-21swapper/111:35:011
4161116994134,3cyclictest0-21swapper/110:45:441
4161116994134,3cyclictest0-21swapper/110:45:441
4161116994132,6cyclictest420-21avahi-daemon11:43:371
4161115994131,4cyclictest0-21swapper/010:29:310
4161115994130,7cyclictest45307-21unixbench-2d10:00:480
416111799407,3cyclictest21148-21ssh09:30:512
416111799405,9cyclictest100771-21ssh11:10:362
416111799403,5cyclictest964-21ntpq09:00:352
4161117994029,7cyclictest420-21avahi-daemon08:20:322
4161117994024,12cyclictest0-21swapper/209:25:002
416111799401,9cyclictest64045-21ssh10:25:052
416111799401,9cyclictest4178465-21ntpq08:05:362
416111799401,9cyclictest116977-21ssh11:32:192
4161117994018,19cyclictest4169980-21unixbench_multi07:35:472
416111799401,6cyclictest4193465-21ntpq08:55:402
416111799401,15cyclictest96979-21ntpq11:05:392
416111799401,13cyclictest77109-21ntpq10:40:392
416111799401,13cyclictest51691-21sed10:10:012
416111799401,13cyclictest122184-21aten_r2power_po11:40:282
416111799401,12cyclictest4166475-21ntpq07:25:402
416111799401,11cyclictest131607-21ssh11:52:572
416111799401,10cyclictest83059-21ssh10:48:302
416111799401,10cyclictest83059-21ssh10:48:302
416111799401,10cyclictest4427-21ntp_states09:10:422
416111799401,10cyclictest26450-21ssh09:38:202
416111799401,10cyclictest156520-21ssh12:25:272
416111799401,10cyclictest144115-21ssh12:09:452
4161116994032,5cyclictest1-21systemd12:00:011
4161116994032,4cyclictest0-21swapper/112:35:471
4161116994032,4cyclictest0-21swapper/112:35:471
4161116994031,6cyclictest49212-21threads10:05:491
4161116994031,4cyclictest0-21swapper/111:50:361
4161116994029,7cyclictest0-21swapper/112:15:371
4161115994030,7cyclictest75873-21munin-plugin-st10:40:020
4161115994024,13cyclictest4356-21ssh09:10:390
4161115994024,11cyclictest0-21swapper/011:10:370
416111799393,7cyclictest4172521-21ntpq07:45:372
416111799392,8cyclictest36745-21ntp_states09:50:412
416111799392,5cyclictest4181478-21ntpq08:15:382
416111799392,5cyclictest4174016-21ntpq07:50:392
416111799392,5cyclictest4163568-21ntpq07:15:422
416111799391,9cyclictest72132-21ssh10:35:102
416111799391,9cyclictest107604-21ssh11:20:112
416111799391,8cyclictest4171076-21ntpq07:40:412
416111799391,8cyclictest4162029-21ntpq07:10:392
416111799391,13cyclictest4192105-21ps08:50:432
416111799391,10cyclictest4190412-21/usr/sbin/munin08:45:342
416111799391,10cyclictest136746-21ssh12:00:102
4161116993931,5cyclictest634-21fschecks_count09:00:311
4161116993931,5cyclictest111737-21fschecks_time11:25:321
4161116993931,4cyclictest71443-21kworker/u8:16-xprtiod11:10:011
416111599398,25cyclictest69002-21ssh10:30:370
4161115993931,5cyclictest4177741-21latency_hist08:05:030
4161115993928,8cyclictest122557-21latency11:40:330
416111799382,16cyclictest261-21systemd-journal07:55:382
416111799381,8cyclictest4184578-21ntp_states08:25:422
4161117993817,18cyclictest261-21systemd-journal08:35:012
416111699387,24cyclictest85073-21ssh10:50:401
4161116993831,3cyclictest0-21swapper/112:10:421
4161116993828,7cyclictest261-21systemd-journal11:24:281
4161115993829,6cyclictest137388-21latency12:00:310
4161115993828,7cyclictest420-21avahi-daemon10:17:250
4161116993729,3cyclictest0-21swapper/110:00:401
4161116993728,4cyclictest0-21swapper/111:00:381
416111699372,29cyclictest52934-21ssh10:10:401
4161115993729,5cyclictest439-21systemd-logind08:15:010
4161115993728,5cyclictest0-21swapper/011:35:020
4161115993728,5cyclictest0-21swapper/009:58:500
4161115993728,5cyclictest0-21swapper/009:58:500
4161115993728,4cyclictest0-21swapper/011:05:370
4161115993726,8cyclictest4176093-21cut08:00:020
416111599372,15cyclictest119257-21awk11:35:380
416111799368,3cyclictest1-21systemd07:10:012
416111799368,3cyclictest1-21systemd07:10:012
416111799361,10cyclictest2597-21ntpq09:05:422
4161116993628,5cyclictest13057-21unixbench_multi09:20:501
4161116993628,5cyclictest125741-21aten_r2power_po11:45:271
4161116993628,5cyclictest125741-21aten_r2power_po11:45:271
4161116993626,6cyclictest91772-21idleruntime-cro11:00:001
4161116993616,6cyclictest104242-21uniq11:15:301
416111599363,13cyclictest1-21systemd08:15:370
4161115993631,3cyclictest0-21swapper/009:25:330
4161115993630,4cyclictest91563-21kworker/u8:6-rpciod11:15:270
4161115993628,4cyclictest0-21swapper/012:15:460
4161115993625,8cyclictest89223-21wc10:55:490
4161115993624,6cyclictest0-21swapper/010:53:210
416111599362,29cyclictest28662-21ssh09:40:420
4161116993528,4cyclictest0-21swapper/107:10:371
4161116993527,5cyclictest63810-21ssh10:25:011
4161116993527,5cyclictest421-21dbus-daemon09:10:021
4161116993527,5cyclictest4173103-21cut07:50:011
4161116993527,5cyclictest1-21systemd08:00:021
4161115993526,6cyclictest3345305-21cups-browsed09:30:130
4161115993521,11cyclictest261-21systemd-journal12:39:370
4161115993521,11cyclictest261-21systemd-journal12:39:370
416111799341,9cyclictest4168078-21ntpq07:30:422
4161116993428,3cyclictest0-21swapper/110:00:011
4161116993428,3cyclictest0-21swapper/110:00:011
4161116993427,4cyclictest8775-21switchtime09:15:481
4161116993427,3cyclictest0-21swapper/109:30:431
4161116993425,6cyclictest421-21dbus-daemon09:26:551
4161116993424,8cyclictest4191896-21needreboot08:50:351
4161116993423,8cyclictest4194081-21idleruntime-cro09:00:011
4161116993422,8cyclictest28125-21fschecks_time09:40:301
416111699342,26cyclictest161309-21ssh12:30:411
4161115993427,4cyclictest156248-21latency_hist12:25:030
4161115993423,8cyclictest0-21swapper/007:20:000
416111599342,27cyclictest141466-21ssh12:05:360
4161116993326,3cyclictest0-21swapper/108:35:371
4161116993325,3cyclictest0-21swapper/110:35:301
4161116993324,6cyclictest4189623-21systemd-run08:45:011
4161116993324,6cyclictest4189623-21systemd-run08:45:011
4161115993326,4cyclictest4177939-21aten_r2power_fr08:05:270
4161115993321,7cyclictest516-21in:imuxsock11:20:360
4161116993226,3cyclictest0-21swapper/107:55:001
4161116993224,5cyclictest4048-21kernelversion09:10:331
4161116993224,3cyclictest0-21swapper/112:25:451
4161115993225,5cyclictest4189549-21grep08:40:480
4161115993225,5cyclictest4189549-21grep08:40:480
4161115993224,4cyclictest0-21swapper/008:35:370
4161115993223,6cyclictest3345305-21cups-browsed11:58:290
416111599321,15cyclictest77149-21grep10:40:400
4161116993124,4cyclictest4165176-21sendmail_mailqu07:20:431
4161116993124,3cyclictest0-21swapper/107:30:391
416111699312,23cyclictest141444-21ssh12:05:351
4161115993118,10cyclictest0-21swapper/011:55:000
4161115993116,12cyclictest695-21cron08:25:010
4161115993113,16cyclictest0-21swapper/012:30:300
416111699303,20cyclictest119227-21ssh11:35:371
4161116993022,5cyclictest421-21dbus-daemon08:25:011
4161116993022,5cyclictest4171532-21sed07:40:491
4161116993022,5cyclictest4166014-21aten2.4-expect07:25:301
4161116993021,6cyclictest1-21systemd08:50:011
4161115993022,5cyclictest4173042-21unixbench_singl07:45:460
416111599302,20cyclictest36640-21sshd09:50:370
4161115993019,8cyclictest156526-21/usr/sbin/munin12:25:310
4161116992923,4cyclictest1055-21mta-sts-daemon08:15:291
4161116992922,5cyclictest4185131-21latency_hist08:30:021
4161116992920,6cyclictest420-21avahi-daemon08:05:401
4161115992923,4cyclictest4191373-21aten_r2power_en08:50:250
4161115992915,10cyclictest12350irq/32-eth007:50:380
4161115992820,5cyclictest4165804-21latency_hist07:25:030
416111599281,9cyclictest171rcu_preempt08:30:430
4161115992819,6cyclictest1055-21mta-sts-daemon08:25:510
416111599273,6cyclictest171rcu_preempt07:35:380
4161115992710,12cyclictest0-21swapper/007:40:480
4161116992619,4cyclictest4164099-21sendmail07:20:001
4161116992619,4cyclictest1056-21mta-sts-daemon08:30:381
4161116992618,5cyclictest4179614-21fschecks_count08:10:311
4161115992619,4cyclictest4194076-21cron09:00:010
416111699256,13cyclictest428-21mta-sts-daemon07:35:391
4161116992517,5cyclictest1054-21mta-sts-daemon08:00:431
416111599257,8cyclictest261-21systemd-journal09:10:010
416111599251,5cyclictest171rcu_preempt07:10:370
416111599241,20cyclictest4190121-21fschecks_time08:45:300
4161115992411,10cyclictest359-21aten_r2power_cu09:00:270
416111599232,15cyclictest4166569-21ntpq07:25:450
4161115992317,4cyclictest4167633-21fschecks_time07:30:300
4161115992216,3cyclictest0-21swapper/007:10:000
4161115992216,3cyclictest0-21swapper/007:10:000
4161116991913,3cyclictest0-21swapper/107:10:011
4161116991913,3cyclictest0-21swapper/107:10:011
416111899172,8cyclictest0-21swapper/310:00:433
416111899171,8cyclictest0-21swapper/309:20:403
416111899171,7cyclictest0-21swapper/307:55:393
416111899171,6cyclictest0-21swapper/309:05:403
416111899163,5cyclictest0-21swapper/310:35:343
416111899162,8cyclictest0-21swapper/311:10:373
416111899161,7cyclictest0-21swapper/308:25:353
416111899153,8cyclictest0-21swapper/310:05:373
416111899151,8cyclictest0-21swapper/312:35:383
416111899151,8cyclictest0-21swapper/312:35:383
416111899151,7cyclictest0-21swapper/311:20:433
416111899142,7cyclictest0-21swapper/310:25:403
416111899142,7cyclictest0-21swapper/309:50:433
416111899142,6cyclictest0-21swapper/312:30:383
416111899142,6cyclictest0-21swapper/312:00:423
416111899142,6cyclictest0-21swapper/309:00:363
416111899142,5cyclictest0-21swapper/307:10:413
416111899141,7cyclictest0-21swapper/310:40:403
416111899141,7cyclictest0-21swapper/309:55:403
416111899141,7cyclictest0-21swapper/309:55:403
416111899141,7cyclictest0-21swapper/309:25:403
416111899141,7cyclictest0-21swapper/308:30:403
416111899141,6cyclictest0-21swapper/309:45:363
416111899141,6cyclictest0-21swapper/308:50:363
416111899141,6cyclictest0-21swapper/307:40:393
416111899134,6cyclictest0-21swapper/311:25:373
416111899132,7cyclictest0-21swapper/311:50:373
416111899132,7cyclictest0-21swapper/310:20:323
416111899132,6cyclictest0-21swapper/309:10:433
416111899132,6cyclictest0-21swapper/308:40:013
416111899132,5cyclictest0-21swapper/312:20:293
416111899132,5cyclictest0-21swapper/307:35:353
416111899132,4cyclictest0-21swapper/308:00:363
416111899131,7cyclictest0-21swapper/311:45:383
416111899131,7cyclictest0-21swapper/311:45:383
416111899131,7cyclictest0-21swapper/311:30:383
416111899131,7cyclictest0-21swapper/311:00:453
416111899131,7cyclictest0-21swapper/310:50:403
416111899131,7cyclictest0-21swapper/310:30:383
416111899131,7cyclictest0-21swapper/309:15:433
416111899131,7cyclictest0-21swapper/308:55:393
416111899131,7cyclictest0-21swapper/307:30:353
416111899131,6cyclictest0-21swapper/308:20:433
416111899123,5cyclictest0-21swapper/308:15:373
416111899122,6cyclictest0-21swapper/312:10:293
416111899121,7cyclictest0-21swapper/311:40:433
416111899121,7cyclictest0-21swapper/310:55:293
416111899121,7cyclictest0-21swapper/310:10:403
416111899121,7cyclictest0-21swapper/308:40:373
416111899121,7cyclictest0-21swapper/308:40:373
416111899121,6cyclictest0-21swapper/310:45:363
416111899121,6cyclictest0-21swapper/310:45:363
416111899121,6cyclictest0-21swapper/308:10:363
416111899121,5cyclictest0-21swapper/307:45:353
416111899113,5cyclictest0-21swapper/311:05:393
416111899112,5cyclictest0-21swapper/312:05:323
416111899112,4cyclictest0-21swapper/307:25:393
416111899111,7cyclictest0-21swapper/310:15:363
416111899111,6cyclictest0-21swapper/312:15:323
416111899111,6cyclictest0-21swapper/311:35:283
416111899111,6cyclictest0-21swapper/309:35:313
416111899111,6cyclictest0-21swapper/308:05:433
416111899111,6cyclictest0-21swapper/307:50:373
416111899101,6cyclictest0-21swapper/311:15:263
416111899101,6cyclictest0-21swapper/309:40:283
416111899101,5cyclictest0-21swapper/311:55:153
416111899101,5cyclictest0-21swapper/309:30:403
416111899101,5cyclictest0-21swapper/308:45:303
416111899101,5cyclictest0-21swapper/307:20:353
416111899101,5cyclictest0-21swapper/307:15:363
41611189992,5cyclictest0-21swapper/312:25:293
41611189981,4cyclictest0-21swapper/307:10:023
41611189981,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional