You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-02 - 14:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sun Aug 02, 2026 12:43:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2503861996354,5cyclictest2524848-21idleruntime-cro08:20:011
2503861995242,7cyclictest2545953-21munin-run09:20:001
2503860995246,4cyclictest2506853-21idleruntime-cro07:20:010
250386099492,40cyclictest151ktimers/008:20:510
2503862994736,8cyclictest2582998-21acpi10:20:402
2503861994737,7cyclictest2521327-21sshd08:06:011
2503862994538,3cyclictest0-21swapper/209:30:562
2503861994536,6cyclictest439-21systemd-logind08:30:011
2503861994536,6cyclictest439-21systemd-logind08:30:011
2503860994530,12cyclictest2647268-21fschecks_time12:05:430
2503862994434,3cyclictest0-21swapper/207:35:542
2503862994432,8cyclictest2625770-21aten_r2power_po11:30:402
2503861994436,5cyclictest2539840-21munin-plugin-st09:10:011
2503861994436,5cyclictest2508376-21systemctl07:25:011
2503861994436,5cyclictest2506879-21systemd-run07:20:011
2503861994436,5cyclictest1003-21mta-sts-daemon10:51:521
2503862994333,6cyclictest2580539-21needreboot10:15:482
2503861994334,6cyclictest2589975-21mailstats10:30:571
2503861994334,6cyclictest2565643-21proc_pri09:51:001
2503860994333,7cyclictest2592286-21cpu10:35:420
2503862994233,6cyclictest2631280-21cron11:40:012
2503862994222,7cyclictest171rcu_preempt11:00:572
2503862994222,7cyclictest171rcu_preempt11:00:572
2503861994234,5cyclictest2605558-21unixbench_multi10:56:021
2503861994232,6cyclictest2594885-21latency_hist10:40:021
250386099423,29cyclictest2541036-21ssh09:10:570
2503860994232,3cyclictest0-21swapper/009:55:580
2503861994133,5cyclictest1-21systemd10:10:021
2503861994132,6cyclictest421-21dbus-daemon11:25:021
2503861994132,4cyclictest2590716-21kworker/u8:4-xprtiod11:01:031
2503861994132,4cyclictest2590716-21kworker/u8:4-xprtiod11:01:031
2503860994130,6cyclictest12650irq/32-eth010:25:490
2503862994031,5cyclictest2545952-21sendmail09:20:002
2503862994028,9cyclictest2541552-21unixbench_multi09:11:072
2503861994032,5cyclictest2539835-21diskmemload09:20:591
2503860994034,3cyclictest2539954-21latency_hist09:10:030
2503860994030,8cyclictest2575184-21unixbench_singl10:06:030
2503862993932,3cyclictest0-21swapper/208:00:012
2503862993931,5cyclictest2619107-21munin-plugin-st11:20:002
2503862993931,5cyclictest2597889-21latency_hist10:45:022
2503862993928,8cyclictest2536824-21munin-plugin-st09:00:012
2503862993928,8cyclictest2536824-21munin-plugin-st09:00:012
2503861993933,3cyclictest0-21swapper/110:26:021
250386199393,27cyclictest2626329-21ssh11:30:491
2503861993931,5cyclictest2634913-21aten_r2power_po11:45:401
2503860993933,3cyclictest0-21swapper/007:35:010
2503860993920,15cyclictest151ktimers/010:40:510
250386299383,25cyclictest2589763-21ssh10:30:502
2503862993829,6cyclictest2625238-21latency_hist11:30:022
2503862993829,5cyclictest2549018-21sh09:25:002
2503862993827,8cyclictest2524838-21munin-run08:20:012
250386199384,4cyclictest0-21swapper/108:35:511
2503861993831,3cyclictest0-21swapper/112:15:021
2503861993831,3cyclictest0-21swapper/109:40:581
2503861993830,4cyclictest1002-21mta-sts-daemon10:19:591
2503861993828,7cyclictest2623521-21mailstats11:25:571
2503861993827,7cyclictest1727784-21cupsd11:13:101
2503861993827,7cyclictest1727784-21cupsd11:13:101
2503861993827,6cyclictest1001-21mta-sts-daemon12:37:581
2503861993827,6cyclictest1001-21mta-sts-daemon12:37:581
2503860993830,5cyclictest2626895-21unixbench_multi11:31:010
2503860993830,5cyclictest1727787-21cups-browsed08:20:010
2503860993828,7cyclictest2666154-21mailstats12:35:580
2503860993828,7cyclictest2666154-21mailstats12:35:580
2503860993828,7cyclictest2571382-21missed_timers10:00:480
2503860993825,4cyclictest2558204-21cron09:40:010
2503862993730,4cyclictest2634357-21kworker/u8:1-rpciod11:56:032
2503862993727,6cyclictest2656981-21proc_pri12:20:582
2503862993727,6cyclictest0-21swapper/212:15:592
250386299371,3cyclictest171rcu_preempt10:25:562
2503861993730,3cyclictest0-21swapper/112:05:491
2503861993729,5cyclictest2525382-21fschecks_time08:20:451
2503860993731,3cyclictest0-21swapper/011:55:560
2503860993730,5cyclictest439-21systemd-logind10:50:020
2503860993730,4cyclictest0-21swapper/012:15:010
2503862993631,2cyclictest0-21swapper/209:10:022
2503862993628,5cyclictest2611667-21grep11:06:022
2503862993628,5cyclictest2611667-21grep11:06:022
2503862993628,5cyclictest2570378-21munin-run10:00:012
2503862993628,5cyclictest2506759-21unixbench_singl07:16:022
2503862993627,6cyclictest2573452-21sh10:05:012
2503861993628,4cyclictest0-21swapper/110:10:581
2503861993627,5cyclictest0-21swapper/109:45:541
2503861993625,8cyclictest2535329-21/usr/sbin/munin08:55:021
250386199360,7cyclictest2539835-21diskmemload10:40:481
2503860993629,5cyclictest2503865-21systemd-run07:10:010
2503860993627,4cyclictest0-21swapper/009:40:590
2503860993623,10cyclictest2622165-21munin-run11:25:010
2503862993528,3cyclictest0-21swapper/210:35:522
2503862993527,5cyclictest2526374-21systemd-run08:25:022
2503862993522,9cyclictest2574173-21uniq10:05:432
2503862993522,5cyclictest1001-21mta-sts-daemon09:40:592
2503861993529,3cyclictest0-21swapper/112:31:021
2503861993529,3cyclictest0-21swapper/111:55:021
2503861993527,4cyclictest0-21swapper/111:20:011
2503861993527,4cyclictest0-21swapper/111:05:561
2503861993527,4cyclictest0-21swapper/111:05:561
2503861993520,10cyclictest421-21dbus-daemon11:37:001
2503860993528,4cyclictest2638705-21proc_pri11:51:000
2503860993528,4cyclictest1-21systemd07:15:010
2503860993527,6cyclictest2536822-21sendmail09:00:010
2503860993527,6cyclictest2536822-21sendmail09:00:010
2503860993527,5cyclictest2532366-21systemd-run08:45:020
2503860993526,7cyclictest2589642-21missed_timers10:30:470
2503860993526,6cyclictest12650irq/32-eth010:10:490
2503860993524,9cyclictest2656775-21gcc12:20:500
250386099352,24cyclictest2601935-21ssh10:50:510
2503862993428,3cyclictest0-21swapper/210:50:012
2503862993426,5cyclictest2521840-21munin-plugin-st08:10:012
2503862993426,5cyclictest0-21swapper/212:11:022
250386299342,28cyclictest2620340-21ssh11:20:542
250386199343,27cyclictest2571604-21ssh10:00:561
2503861993428,3cyclictest0-21swapper/112:30:021
2503861993424,6cyclictest2643550-21latency_hist12:00:021
2503861993424,6cyclictest1727784-21cupsd12:02:001
2503860993427,4cyclictest2555183-21systemd-run09:35:010
2503860993424,7cyclictest2512147-21needreboot07:35:470
2503862993327,3cyclictest0-21swapper/211:00:012
2503862993326,4cyclictest2527863-21systemd-run08:30:012
2503862993326,4cyclictest2527863-21systemd-run08:30:012
2503862993325,5cyclictest2523359-21systemd-run08:15:012
2503862993323,7cyclictest2503859-21cyclictest09:50:022
2503862993323,6cyclictest2637989-21aten_r2power_vo11:50:442
2503862993321,9cyclictest2565274-21latency09:50:492
2503862993317,10cyclictest265-21systemd-journal11:14:022
2503862993317,10cyclictest265-21systemd-journal11:14:022
250386199336,19cyclictest2656792-21ssh12:20:501
2503861993324,6cyclictest2512752-21sed07:36:011
250386199332,25cyclictest2568540-21ssh09:55:541
250386199332,22cyclictest2553295-21ssh09:30:541
250386199331,27cyclictest2556868-21awk09:36:021
250386199330,5cyclictest171rcu_preempt11:40:581
2503860993327,4cyclictest1727784-21cupsd12:25:070
2503860993326,5cyclictest2662953-21ntp_offset12:30:570
2503860993325,3cyclictest0-21swapper/011:15:560
2503860993324,6cyclictest12650irq/32-eth009:20:530
2503862993225,5cyclictest0-21swapper/208:05:012
2503862993224,5cyclictest2603860-21cron10:55:012
2503862993222,6cyclictest2556517-21systemctl09:35:582
2503862993222,4cyclictest0-21swapper/212:25:502
2503862993221,8cyclictest2529333-21/usr/sbin/munin08:35:012
2503861993224,5cyclictest2517337-21munin-plugin-st07:55:011
2503861993216,3cyclictest0-21swapper/109:25:521
2503861993212,15cyclictest1727787-21cups-browsed07:10:541
2503860993225,4cyclictest2521779-21head08:06:070
2503860993221,8cyclictest0-21swapper/011:40:020
2503862993126,2cyclictest0-21swapper/208:40:012
2503862993125,3cyclictest0-21swapper/207:30:002
2503862993125,3cyclictest0-21swapper/207:30:002
2503862993124,4cyclictest1002-21mta-sts-daemon11:45:282
2503861993125,3cyclictest0-21swapper/109:15:011
2503861993123,5cyclictest2598753-21missed_timers10:45:461
250386199311,20cyclictest1727787-21cups-browsed07:40:521
2503860993125,4cyclictest2514235-21unixbench_multi07:41:020
2503860993124,5cyclictest2533849-21systemd-run08:50:010
2503860993124,4cyclictest2565018-21sh09:50:450
2503860993117,9cyclictest151ktimers/010:15:570
2503862993021,6cyclictest2550735-21unixbench_multi09:26:032
2503862993021,4cyclictest0-21swapper/210:10:562
2503862993016,7cyclictest2644754-21sh12:00:592
250386299301,23cyclictest2662936-21ssh12:30:562
2503861993022,5cyclictest0-21swapper/108:00:011
2503861993019,2cyclictest2515843-21munin-run07:50:011
2503860993023,3cyclictest0-21swapper/011:40:400
2503860993020,7cyclictest2637389-21grep11:50:020
2503860993019,5cyclictest1000-21mta-sts-daemon10:20:510
250386299292,20cyclictest665-21snmpd12:35:532
250386299292,20cyclictest665-21snmpd12:35:532
2503862992920,5cyclictest421-21dbus-daemon07:25:012
2503861992923,3cyclictest0-21swapper/112:15:531
2503861992922,3cyclictest0-21swapper/110:20:531
2503861992920,5cyclictest0-21swapper/108:56:021
2503861992920,5cyclictest0-21swapper/108:56:021
2503861992919,7cyclictest2537935-21sendmail_mailtr09:01:011
2503861992919,7cyclictest2510514-21latency07:30:451
2503861992914,3cyclictest0-21swapper/108:40:451
2503860992924,3cyclictest2503859-21cyclictest09:45:500
2503860992924,3cyclictest1003-21mta-sts-daemon11:07:430
2503860992924,3cyclictest1003-21mta-sts-daemon11:07:430
2503860992923,3cyclictest0-21swapper/011:10:480
2503860992923,3cyclictest0-21swapper/011:10:480
2503860992920,4cyclictest0-21swapper/008:30:520
2503860992919,7cyclictest1002-21mta-sts-daemon11:30:020
250386299282,20cyclictest2632440-21ssh11:40:522
2503862992820,5cyclictest2647119-21ssh12:05:402
250386199282,8cyclictest171rcu_preempt08:45:501
2503861992821,3cyclictest0-21swapper/108:05:011
2503860992822,4cyclictest2545941-21cron09:20:000
250386099281,22cyclictest2605001-21ssh10:55:510
2503862992720,4cyclictest2505332-21sh07:15:002
2503862992715,9cyclictest694-21cron07:55:012
2503860992722,3cyclictest2637513-21kworker/u8:3-xprtiod12:05:010
2503860992722,3cyclictest0-21swapper/009:30:010
2503860992722,2cyclictest0-21swapper/012:15:410
2503860992721,4cyclictest12650irq/32-eth011:00:510
2503860992721,4cyclictest12650irq/32-eth011:00:510
2503860992718,3cyclictest1003-21mta-sts-daemon07:45:280
2503862992619,3cyclictest0-21swapper/207:46:012
2503861992618,5cyclictest2522126-21aten_r2power_en08:10:401
250386099262,16cyclictest1727784-21cupsd08:25:520
250386099262,16cyclictest1727784-21cupsd08:25:520
2503860992619,4cyclictest2509751-21unixbench_multi07:26:010
2503860992619,4cyclictest2509751-21unixbench_multi07:26:010
2503860992619,4cyclictest0-21swapper/008:00:010
2503862992517,5cyclictest1727787-21cups-browsed07:43:272
250386199255,13cyclictest428-21mta-sts-daemon07:25:541
250386199255,13cyclictest428-21mta-sts-daemon07:25:541
2503860992518,5cyclictest1002-21mta-sts-daemon08:35:520
2503860992515,5cyclictest2523360-21munin-run08:15:020
2503862992418,4cyclictest2503859-21cyclictest09:05:012
2503862992417,4cyclictest2510365-21fschecks_count07:30:422
2503862992414,7cyclictest485-21in:imuxsock08:45:012
2503861992416,6cyclictest2528137-21aten_r2power_en08:30:411
2503861992413,4cyclictest2501200-21kworker/u8:14-xprtiod07:10:011
250386099246,12cyclictest151ktimers/008:50:510
250386099242,8cyclictest2537643-21needreboot09:00:470
2503860992410,3cyclictest0-21swapper/007:20:480
2503862992311,10cyclictest351ktimers/208:45:462
250386099232,16cyclictest2516811-21ntp_states07:50:570
2503862992214,6cyclictest2534162-21aten_r2power_fr08:50:402
2503860992218,2cyclictest0-21swapper/008:01:010
2503862991913,3cyclictest0-21swapper/207:10:022
250386399171,8cyclictest0-21swapper/309:50:593
250386399161,7cyclictest0-21swapper/308:15:473
250386399161,6cyclictest0-21swapper/308:10:563
250386399153,7cyclictest0-21swapper/312:16:043
250386399153,6cyclictest0-21swapper/310:05:543
250386399151,8cyclictest0-21swapper/311:10:483
250386399151,8cyclictest0-21swapper/311:10:483
250386399151,7cyclictest0-21swapper/310:30:403
250386399151,7cyclictest0-21swapper/309:30:543
250386399150,5cyclictest0-21swapper/311:15:573
250386399143,7cyclictest0-21swapper/312:35:563
250386399143,7cyclictest0-21swapper/312:35:563
250386399142,6cyclictest0-21swapper/310:35:493
250386399142,6cyclictest0-21swapper/310:25:543
250386399142,6cyclictest0-21swapper/307:30:013
250386399142,6cyclictest0-21swapper/307:30:013
250386399141,7cyclictest0-21swapper/311:30:513
250386399141,7cyclictest0-21swapper/310:40:483
250386399141,7cyclictest0-21swapper/307:10:563
250386399141,6cyclictest0-21swapper/312:05:463
250386399141,6cyclictest0-21swapper/308:30:563
250386399141,6cyclictest0-21swapper/308:00:013
250386399141,6cyclictest0-21swapper/307:45:543
250386399141,6cyclictest0-21swapper/307:35:493
250386399141,4cyclictest0-21swapper/309:40:523
250386399133,6cyclictest0-21swapper/311:50:013
250386399132,7cyclictest0-21swapper/312:10:583
250386399132,7cyclictest0-21swapper/311:40:403
250386399132,7cyclictest0-21swapper/311:00:503
250386399132,7cyclictest0-21swapper/311:00:503
250386399132,6cyclictest0-21swapper/310:10:483
250386399132,6cyclictest0-21swapper/309:35:543
250386399131,7cyclictest0-21swapper/309:10:543
250386399131,7cyclictest0-21swapper/308:45:483
250386399131,5cyclictest0-21swapper/309:05:503
250386399131,4cyclictest0-21swapper/310:45:533
250386399130,6cyclictest0-21swapper/308:50:503
250386399123,7cyclictest0-21swapper/312:25:423
250386399123,7cyclictest0-21swapper/311:35:443
250386399122,6cyclictest0-21swapper/311:25:433
250386399122,6cyclictest0-21swapper/311:06:013
250386399122,6cyclictest0-21swapper/311:06:013
250386399122,6cyclictest0-21swapper/310:15:513
250386399122,6cyclictest0-21swapper/310:00:553
250386399122,6cyclictest0-21swapper/310:00:023
250386399122,6cyclictest0-21swapper/309:45:443
250386399122,6cyclictest0-21swapper/309:15:433
250386399122,5cyclictest0-21swapper/309:00:583
250386399122,5cyclictest0-21swapper/308:45:023
250386399122,4cyclictest0-21swapper/308:00:513
250386399121,7cyclictest0-21swapper/311:50:483
250386399121,7cyclictest0-21swapper/310:55:433
250386399121,7cyclictest0-21swapper/309:25:523
250386399121,6cyclictest0-21swapper/312:20:403
250386399121,6cyclictest0-21swapper/308:40:023
250386399121,6cyclictest0-21swapper/308:05:583
250386399121,6cyclictest0-21swapper/307:30:403
250386399121,5cyclictest0-21swapper/307:15:503
250386399113,5cyclictest0-21swapper/308:20:523
250386399113,4cyclictest0-21swapper/307:20:553
250386399112,5cyclictest0-21swapper/311:20:413
250386399112,5cyclictest0-21swapper/310:20:413
250386399111,6cyclictest0-21swapper/312:30:463
250386399111,6cyclictest0-21swapper/311:55:413
250386399111,6cyclictest0-21swapper/310:50:553
250386399111,6cyclictest0-21swapper/309:20:513
250386399111,6cyclictest0-21swapper/308:55:573
250386399111,6cyclictest0-21swapper/308:55:573
250386399102,5cyclictest0-21swapper/312:01:023
25038639993,4cyclictest0-21swapper/307:50:503
25038639992,5cyclictest0-21swapper/308:30:023
25038639992,5cyclictest0-21swapper/308:30:023
25038639991,7cyclictest0-21swapper/307:10:013
25038639991,5cyclictest0-21swapper/307:40:433
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional