You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 00:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Mon Jul 13, 2026 12:43:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2764908996735,17cyclictest1-21systemd07:30:011
2764907996546,9cyclictest12650irq/32-eth011:15:420
2764908994919,25cyclictest2876833-21missed_timers11:15:381
2764908994810,32cyclictest2805039-21ssh09:15:421
2764908994736,6cyclictest0-21swapper/110:25:471
2764907994739,5cyclictest2868604-21head11:00:530
2764908994636,6cyclictest1961693-21cups-browsed10:24:521
276490999453,34cyclictest2859018-21ssh10:45:422
2764908994535,6cyclictest2905830-21debian-sa112:05:011
2764907994529,13cyclictest1961693-21cups-browsed10:40:280
2764909994434,6cyclictest421-21dbus-daemon10:20:022
2764909994434,6cyclictest421-21dbus-daemon10:20:022
2764908994434,7cyclictest2764913-21systemd-run07:10:011
276490899441,15cyclictest2798763-21ntpq09:00:421
276490799443,36cyclictest151ktimers/011:55:480
2764907994425,16cyclictest2888659-21kernelversion11:35:340
2764907994425,16cyclictest2888659-21kernelversion11:35:340
276490999435,32cyclictest421-21dbus-daemon09:00:492
276490899432,7cyclictest2791301-21ntpq08:35:411
2764908994321,19cyclictest0-21swapper/111:05:481
2764909994233,6cyclictest2834676-21/usr/sbin/munin10:05:342
2764909994231,7cyclictest421-21dbus-daemon11:54:562
2764908994225,14cyclictest2808629-21unixbench-2d09:21:031
2764908994225,14cyclictest2808629-21unixbench-2d09:21:031
276490899422,14cyclictest2832126-21ntpq10:00:441
276490899421,16cyclictest2800293-21ntpq09:05:421
276490899421,12cyclictest2906934-21ntpq12:05:451
276490899421,10cyclictest2782306-21ntpq08:05:421
276490899421,10cyclictest2777867-21ntpq07:50:471
2764907994230,9cyclictest2803926-21cron09:15:010
2764907994230,10cyclictest2914810-21idleruntime-cro12:20:010
2764909994133,5cyclictest2797911-21systemd-run09:00:022
2764909994132,6cyclictest2819000-21cron09:40:012
2764909994131,6cyclictest2863908-21grep10:55:012
2764909994131,6cyclictest2863908-21grep10:55:012
2764909994131,4cyclictest0-21swapper/211:15:422
2764908994132,5cyclictest0-21swapper/112:40:011
2764908994129,8cyclictest2814457-21timerandwakeup09:30:511
276490899412,6cyclictest2918999-21ntpq12:25:451
276490899412,6cyclictest2909974-21ntpq12:10:441
276490899412,6cyclictest2820142-21ntpq09:40:431
276490899412,6cyclictest2771811-21ntpq07:30:421
276490899412,6cyclictest2771811-21ntpq07:30:421
276490899411,13cyclictest2893142-21ssh11:42:461
276490899411,13cyclictest2893142-21ssh11:42:461
276490899411,10cyclictest2774811-21ntpq07:40:411
276490899411,10cyclictest2774811-21ntpq07:40:411
2764907994134,4cyclictest421-21dbus-daemon09:35:010
276490799413,29cyclictest12650irq/32-eth011:40:480
276490799413,29cyclictest12650irq/32-eth011:40:480
2764907994114,22cyclictest151ktimers/009:40:420
2764909994032,5cyclictest1001-21mta-sts-daemon10:35:422
2764909994030,4cyclictest0-21swapper/212:35:432
276490899404,7cyclictest2797293-21ntpq08:55:431
2764908994030,5cyclictest0-21swapper/109:35:391
2764908994029,7cyclictest421-21dbus-daemon10:38:491
276490899402,7cyclictest2802037-21ntpq09:10:441
276490899402,5cyclictest2873950-21ntpq11:10:371
2764908994020,17cyclictest2823358-21mailstats09:45:491
276490899401,5cyclictest2921928-21ntpq12:30:441
276490899401,13cyclictest2837854-21grep10:10:361
276490899400,9cyclictest2792122-21/usr/sbin/munin08:40:521
276490899400,9cyclictest2792122-21/usr/sbin/munin08:40:521
2764909993931,5cyclictest2810000-21systemd-run09:25:022
2764909993931,5cyclictest2810000-21systemd-run09:25:022
2764908993931,5cyclictest2894253-21aten_r2power_cu11:45:311
276490899392,8cyclictest2902812-21munin-run12:00:011
276490899392,6cyclictest2795771-21ntpq08:50:411
276490899392,4cyclictest2811049-21ntpq09:25:411
276490899392,4cyclictest2811049-21ntpq09:25:411
276490899392,4cyclictest2785272-21ntpq08:15:421
276490899392,4cyclictest2773242-21ntpq07:35:381
276490899392,4cyclictest2773242-21ntpq07:35:381
276490899392,28cyclictest2888866-21ssh11:35:371
276490899392,28cyclictest2888866-21ssh11:35:371
2764908993921,15cyclictest2899825-21munin-run11:55:011
276490899391,5cyclictest2767341-21ntpq07:15:461
276490899391,4cyclictest2768862-21ntp_states07:20:531
276490899390,4cyclictest2783761-21ntpq08:10:391
2764907993929,8cyclictest265-21systemd-journal12:25:010
276490899382,7cyclictest2794347-21ntpq08:45:481
276490899382,6cyclictest2916003-21ntpq12:20:441
276490899381,5cyclictest2880069-21ntpq11:20:511
276490899381,4cyclictest2776887-21munin-run07:50:011
276490899381,4cyclictest2765803-21ntpq07:10:401
276490899381,10cyclictest2829197-21ntpq09:55:461
276490899381,10cyclictest2789789-21ntpq08:30:421
276490899380,11cyclictest2887515-21ssh11:33:591
276490899380,11cyclictest2836206-21ssh10:07:261
276490899380,10cyclictest2850016-21ntpq10:30:391
2764907993831,4cyclictest2863895-21cron10:55:010
2764907993831,4cyclictest2863895-21cron10:55:010
2764907993827,9cyclictest2799393-21munin-plugin-st09:05:010
2764909993730,3cyclictest0-21swapper/211:00:422
2764909993729,5cyclictest2846093-21latency_hist10:25:022
276490899372,6cyclictest2826033-21ssh09:50:381
276490899372,5cyclictest2788242-21ntpq08:25:401
276490899372,12cyclictest2859044-21ntpq10:45:431
2764908993719,14cyclictest0-21swapper/108:20:341
276490899371,7cyclictest2841251-21/usr/sbin/munin10:15:491
276490899371,7cyclictest2841251-21/usr/sbin/munin10:15:491
276490899371,5cyclictest2779289-21ntpq07:55:431
276490899371,4cyclictest2865085-21ntpq10:55:441
276490899371,4cyclictest2865085-21ntpq10:55:441
276490899371,3cyclictest2780757-21ntpq08:00:431
2764907993730,3cyclictest0-21swapper/009:35:520
2764907993729,5cyclictest0-21swapper/012:05:020
2764907993728,7cyclictest2886511-21grep11:30:540
2764907993726,7cyclictest2823720-21unixbench_singl09:45:530
2764909993630,3cyclictest0-21swapper/210:25:322
2764909993628,5cyclictest2920792-21cron12:30:012
2764909993628,3cyclictest0-21swapper/212:10:442
276490899364,11cyclictest2856008-21ntpq10:40:391
276490899363,5cyclictest2862030-21ssh10:50:421
276490899363,5cyclictest2862030-21ssh10:50:421
276490899362,13cyclictest2912722-21memory12:15:371
276490899361,6cyclictest2868050-21ntpq11:00:431
2764907993630,4cyclictest2784382-21munin-run08:15:010
2764907993629,3cyclictest1000-21mta-sts-daemon12:27:320
2764907993627,7cyclictest2883026-21ntp_offset11:25:450
2764907993623,9cyclictest2767196-21needreboot07:15:400
276490799362,29cyclictest2879999-21sshd11:20:460
2764909993529,3cyclictest0-21swapper/212:20:292
2764909993528,4cyclictest2773927-21systemd-run07:40:012
2764909993528,4cyclictest2773927-21systemd-run07:40:012
2764909993527,4cyclictest0-21swapper/212:15:412
2764909993526,6cyclictest485-21in:imuxsock07:30:012
276490899350,10cyclictest2882655-21awk11:25:371
2764907993528,4cyclictest2826289-21processes09:50:470
2764907993527,5cyclictest2906213-21/usr/sbin/munin12:05:350
2764907993525,7cyclictest2849892-21missed_timers10:30:370
276490999344,24cyclictest2855960-21ssh10:40:372
2764909993427,4cyclictest2777021-21kworker/u8:7-xprtiod09:20:012
2764909993426,5cyclictest2866462-21sh10:58:372
2764909993426,5cyclictest2866462-21sh10:58:372
2764909993424,8cyclictest2785885-21/usr/sbin/munin08:20:022
276490999341,27cyclictest2826221-21rm09:50:462
2764907993428,4cyclictest2846645-21ssh10:25:350
2764909993326,4cyclictest2789483-21if_eth008:30:352
2764909993325,5cyclictest265-21systemd-journal09:50:012
2764909993323,7cyclictest2891718-21latency11:40:372
2764909993323,7cyclictest2891718-21latency11:40:372
2764909993323,5cyclictest1002-21mta-sts-daemon12:05:432
2764909993322,7cyclictest420-21avahi-daemon12:03:012
2764907993327,3cyclictest0-21swapper/011:00:000
2764907993327,3cyclictest0-21swapper/011:00:000
2764907993326,4cyclictest2806979-21ssh09:20:010
2764907993326,4cyclictest2782888-21munin-plugin-st08:10:010
2764907993325,6cyclictest2779875-21idleruntime-cro08:00:010
2764907993324,6cyclictest2921171-21/usr/sbin/munin12:30:320
2764907993324,6cyclictest1961693-21cups-browsed11:11:380
2764907993322,8cyclictest2832447-21switchtime10:00:530
2764909993226,3cyclictest0-21swapper/207:20:012
2764909993224,5cyclictest2765739-21ntp_kernel_err07:10:372
2764909993224,3cyclictest0-21swapper/211:25:472
2764909993223,6cyclictest1003-21mta-sts-daemon11:09:212
2764909993223,5cyclictest2902826-21idleruntime-cro12:00:012
2764907993227,3cyclictest1002-21mta-sts-daemon12:11:510
2764907993225,4cyclictest2871171-21proc_pri11:05:460
2764907993224,3cyclictest0-21swapper/009:55:420
2764907993215,11cyclictest420-21avahi-daemon10:45:470
276490999315,16cyclictest428-21mta-sts-daemon08:35:392
2764909993123,5cyclictest2881986-21latency_hist11:25:022
2764909993123,5cyclictest2812974-21grep09:30:002
2764909993123,5cyclictest2812974-21grep09:30:002
2764909993123,5cyclictest2778375-21idleruntime-cro07:55:012
2764909993121,4cyclictest0-21swapper/208:20:452
276490799314,21cyclictest428-21mta-sts-daemon10:35:450
2764907993125,3cyclictest2763532-21kworker/u8:10-xprtiod09:20:370
2764907993125,3cyclictest2763532-21kworker/u8:10-xprtiod09:20:370
2764907993124,3cyclictest0-21swapper/009:05:400
2764907993119,9cyclictest0-21swapper/008:55:520
2764907993112,12cyclictest12650irq/32-eth012:35:500
2764909993024,3cyclictest0-21swapper/209:55:342
2764909993024,3cyclictest0-21swapper/209:35:002
2764909993023,4cyclictest2796377-21idleruntime-cro08:55:002
2764909993023,4cyclictest0-21swapper/209:40:382
2764909993023,3cyclictest0-21swapper/211:35:462
2764909993023,3cyclictest0-21swapper/211:35:462
2764909993022,5cyclictest2794810-21unixbench_singl08:45:562
2764909993022,5cyclictest1001-21mta-sts-daemon12:31:192
2764909993021,4cyclictest0-21swapper/210:30:442
2764907993024,4cyclictest2790386-21idleruntime-cro08:35:010
2764907993013,14cyclictest12650irq/32-eth010:05:450
2764909992923,2cyclictest0-21swapper/211:10:442
2764907992919,7cyclictest2810477-21basename09:25:310
2764907992919,7cyclictest2810477-21basename09:25:310
2764907992918,8cyclictest0-21swapper/007:15:010
2764907992916,9cyclictest12650irq/32-eth011:45:440
276490999284,5cyclictest2800885-21diskmemload09:10:402
276490999283,16cyclictest2800885-21diskmemload11:30:392
2764909992822,3cyclictest0-21swapper/210:00:442
2764909992820,5cyclictest2781273-21unixbench_multi08:00:552
2764909992820,5cyclictest1001-21mta-sts-daemon07:20:302
2764907992822,4cyclictest2840004-21idleruntime-cro10:15:010
2764907992821,5cyclictest421-21dbus-daemon08:40:450
2764907992821,5cyclictest421-21dbus-daemon08:40:450
2764907992819,5cyclictest0-21swapper/008:50:010
276490799281,23cyclictest2841119-21ntpq10:15:440
276490799281,23cyclictest2841119-21ntpq10:15:440
2764909992721,3cyclictest0-21swapper/211:50:002
2764909992720,4cyclictest2772304-21unixbench_singl07:30:522
2764909992720,4cyclictest2772304-21unixbench_singl07:30:522
276490799272,4cyclictest171rcu_preempt07:50:450
2764907992720,3cyclictest2875920-21kworker/u8:5-rpciod11:50:410
2764907992719,5cyclictest2781248-21unixbench-2d08:00:550
2764907992712,10cyclictest151ktimers/007:30:460
2764907992712,10cyclictest151ktimers/007:30:460
276490799271,20cyclictest12650irq/32-eth008:25:460
276490999266,6cyclictest171rcu_preempt08:40:422
276490999266,6cyclictest171rcu_preempt08:40:422
276490999265,12cyclictest428-21mta-sts-daemon07:40:442
276490999265,12cyclictest428-21mta-sts-daemon07:40:442
2764909992619,3cyclictest0-21swapper/210:10:402
2764909992618,6cyclictest2775924-21fschecks_time07:45:342
2764907992621,3cyclictest2791918-21unin-run08:40:010
2764909992519,3cyclictest0-21swapper/209:10:022
2764909992519,3cyclictest0-21swapper/208:00:012
2764909992518,4cyclictest2779889-21kworker/u8:4-xprtiod08:05:322
2764907992519,4cyclictest2844443-21ssh10:20:510
2764907992519,4cyclictest2769399-21cron07:25:010
2764907992515,8cyclictest2785031-21kernelversion08:15:370
276490799251,19cyclictest151ktimers/007:45:460
2764907992417,5cyclictest265-21systemd-journal08:21:340
2764907992412,8cyclictest12650irq/32-eth007:40:440
2764907992412,8cyclictest12650irq/32-eth007:40:440
276490999234,12cyclictest2060304-21munin-node08:25:432
2764907992313,6cyclictest12750irq/33-eth007:35:410
2764907992313,6cyclictest12750irq/33-eth007:35:410
276490799231,19cyclictest2795845-21ntpq08:50:450
276490999222,14cyclictest2783742-21ntpq08:10:382
276491099211,8cyclictest0-21swapper/312:25:403
2764909992117,2cyclictest0-21swapper/207:10:022
276490799211,17cyclictest2770305-21ntpq07:25:430
276491099192,8cyclictest0-21swapper/310:40:423
276491099183,9cyclictest0-21swapper/312:10:433
276491099182,8cyclictest0-21swapper/311:30:423
276491099174,5cyclictest0-21swapper/311:20:463
276491099173,7cyclictest0-21swapper/309:10:013
276491099172,8cyclictest0-21swapper/312:00:373
276491099172,8cyclictest0-21swapper/309:55:403
276491099172,8cyclictest0-21swapper/309:30:433
276491099172,7cyclictest0-21swapper/308:50:393
276491099172,6cyclictest0-21swapper/308:35:383
276491099171,10cyclictest0-21swapper/310:25:513
276491099163,8cyclictest0-21swapper/311:40:543
276491099163,8cyclictest0-21swapper/311:40:543
276491099163,8cyclictest0-21swapper/311:00:413
276491099163,8cyclictest0-21swapper/310:45:493
276491099163,8cyclictest0-21swapper/309:15:453
276491099163,7cyclictest0-21swapper/310:55:393
276491099163,7cyclictest0-21swapper/310:55:393
276491099163,7cyclictest0-21swapper/310:05:533
276491099163,5cyclictest0-21swapper/308:00:463
276491099162,9cyclictest0-21swapper/312:15:403
276491099162,8cyclictest0-21swapper/311:35:303
276491099162,8cyclictest0-21swapper/311:35:303
276491099162,8cyclictest0-21swapper/311:15:453
276491099162,8cyclictest0-21swapper/310:20:413
276491099162,7cyclictest0-21swapper/311:10:413
276491099162,7cyclictest0-21swapper/309:50:333
276491099162,6cyclictest0-21swapper/307:25:433
276491099161,9cyclictest0-21swapper/312:05:343
276491099161,7cyclictest0-21swapper/307:50:423
276491099161,6cyclictest0-21swapper/308:55:403
276491099153,7cyclictest0-21swapper/309:35:413
276491099153,6cyclictest0-21swapper/308:30:423
276491099152,8cyclictest0-21swapper/312:20:383
276491099152,8cyclictest0-21swapper/311:46:343
276491099152,8cyclictest0-21swapper/310:00:313
276491099152,8cyclictest0-21swapper/309:10:353
276491099152,7cyclictest0-21swapper/312:30:433
276491099152,7cyclictest0-21swapper/310:50:293
276491099152,7cyclictest0-21swapper/310:50:293
276491099152,7cyclictest0-21swapper/310:10:413
276491099152,6cyclictest0-21swapper/307:55:423
276491099151,8cyclictest0-21swapper/309:20:373
276491099151,8cyclictest0-21swapper/309:20:373
2764907991510,2cyclictest0-21swapper/007:10:010
276491099143,6cyclictest0-21swapper/310:30:293
276491099143,4cyclictest0-21swapper/307:30:433
276491099143,4cyclictest0-21swapper/307:30:433
276491099142,6cyclictest0-21swapper/307:35:393
276491099142,6cyclictest0-21swapper/307:35:393
276491099142,5cyclictest0-21swapper/308:15:413
276491099141,7cyclictest0-21swapper/311:25:323
276491099141,7cyclictest0-21swapper/308:20:463
276491099141,5cyclictest0-21swapper/308:10:373
276491099134,6cyclictest0-21swapper/310:35:383
276491099133,6cyclictest0-21swapper/309:25:363
276491099133,6cyclictest0-21swapper/309:25:363
276491099132,7cyclictest0-21swapper/311:50:433
276491099131,8cyclictest0-21swapper/311:05:313
276491099131,8cyclictest0-21swapper/310:15:403
276491099131,8cyclictest0-21swapper/310:15:403
276491099131,7cyclictest0-21swapper/312:35:303
276491099131,7cyclictest0-21swapper/309:45:323
276491099131,7cyclictest0-21swapper/309:00:433
276491099131,6cyclictest0-21swapper/308:25:403
276491099130,6cyclictest0-21swapper/308:45:433
276491099124,6cyclictest0-21swapper/309:40:323
276491099123,6cyclictest0-21swapper/311:55:493
276491099122,6cyclictest0-21swapper/307:15:523
276491099121,6cyclictest0-21swapper/307:45:473
276491099120,6cyclictest0-21swapper/307:40:423
276491099120,6cyclictest0-21swapper/307:40:423
276491099112,5cyclictest0-21swapper/308:40:343
276491099112,5cyclictest0-21swapper/308:40:343
276491099112,5cyclictest0-21swapper/307:10:383
276491099102,7cyclictest0-21swapper/307:10:013
276491099101,6cyclictest0-21swapper/307:20:343
276491099101,5cyclictest0-21swapper/308:05:323
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional