You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 20:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sat Jun 28, 2025 12:41:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2705565996352,7cyclictest1910230-21cups-browsed07:40:022
2705563994940,6cyclictest2846449-21/usr/sbin/munin11:50:020
2705563994940,6cyclictest2846449-21/usr/sbin/munin11:50:020
2705565994538,4cyclictest2758126-21grep09:35:012
2705565994535,5cyclictest959-21mta-sts-daemon08:52:562
2705563994537,6cyclictest266-21systemd-journal08:40:010
2705564994436,5cyclictest2816628-21systemd-run11:05:011
270556599436,30cyclictest1690079-21ntpd08:17:572
2705565994335,5cyclictest2797391-21cron10:35:012
2705565994333,7cyclictest2819954-21cron11:10:012
270556499435,29cyclictest2867686-21ssh12:18:021
2705564994336,3cyclictest0-21swapper/109:42:591
2705564994335,5cyclictest2741525-21cron09:10:011
2705563994328,8cyclictest12450irq/32-eth009:27:590
2705563994328,12cyclictest2738539-21sendmail09:00:020
2705563994324,13cyclictest12450irq/32-eth011:18:040
2705565994233,6cyclictest2823249-21systemd-run11:15:022
2705563994235,4cyclictest2761655-21wc09:40:020
270556399421,33cyclictest2838487-21ssh11:37:580
270556399421,33cyclictest2838487-21ssh11:37:580
2705565994134,4cyclictest0-21swapper/209:20:012
2705565994134,4cyclictest0-21swapper/209:20:012
2705564994133,5cyclictest2797384-21cron10:35:011
2705564994133,5cyclictest2787168-21unixbench_singl10:18:071
2705564994132,5cyclictest2790015-21unixbench-2d10:23:061
2705563994134,4cyclictest2829161-21unixbench_singl11:23:080
2705563994134,4cyclictest2829161-21unixbench_singl11:23:080
2705563994122,9cyclictest12550irq/33-eth010:07:560
2705563994122,9cyclictest12550irq/33-eth010:07:560
2705565994032,5cyclictest2764857-21idleruntime-cro09:45:012
2705564994032,5cyclictest2879703-21threads12:33:051
2705564994031,6cyclictest657-21snmpd10:37:491
270556599395,29cyclictest2802713-21ssh10:42:592
2705565993931,5cyclictest2880958-21systemd-run12:35:022
2705565993928,7cyclictest2829989-21systemd-run11:25:022
2705565993928,7cyclictest2829989-21systemd-run11:25:022
270556499396,5cyclictest171rcu_preempt11:27:561
2705564993931,5cyclictest2838360-21ntpq11:37:531
2705564993931,5cyclictest2838360-21ntpq11:37:531
2705564993929,7cyclictest2826493-21munin-run11:20:011
2705563993930,6cyclictest2875816-21timerandwakeup12:28:070
2705565993832,4cyclictest0-21swapper/211:05:012
2705565993831,4cyclictest2853464-21cron12:00:012
2705565993830,4cyclictest0-21swapper/210:47:542
2705565993829,5cyclictest2743144-21aten_r2power_cu09:12:422
2705565993828,7cyclictest2721083-21fschecks_count08:02:462
2705565993828,7cyclictest2721083-21fschecks_count08:02:462
2705564993832,3cyclictest0-21swapper/107:20:011
2705564993832,3cyclictest0-21swapper/107:20:011
2705564993830,5cyclictest2851201-21ssh11:57:421
2705564993830,5cyclictest2835772-21head11:33:051
2705564993830,5cyclictest2770109-21needreboot09:52:511
2705564993828,7cyclictest2748422-21kworker/u8:14-xprtiod09:40:011
2705563993823,13cyclictest2741534-21diskmemload10:54:060
2705563993823,13cyclictest2741534-21diskmemload10:54:060
270556399382,29cyclictest2786497-21ssh10:17:540
2705565993729,5cyclictest2867248-21sh12:17:512
2705565993728,6cyclictest1-21systemd09:30:022
2705564993729,5cyclictest2873212-21latency_hist12:25:031
2705564993729,5cyclictest1-21systemd12:40:021
2705564993729,4cyclictest2810259-21cron10:55:001
2705564993729,4cyclictest2810259-21cron10:55:001
2705564993727,7cyclictest2857427-21sh12:05:011
270556399372,30cyclictest2796003-21ssh10:32:550
2705565993630,3cyclictest0-21swapper/208:40:012
2705565993630,2cyclictest0-21swapper/210:22:552
2705565993629,3cyclictest0-21swapper/210:40:012
2705565993628,5cyclictest2711575-21systemd-run07:30:012
2705565993627,5cyclictest2871767-21mailstats12:23:032
2705565993626,7cyclictest2751562-21sh09:25:002
2705564993628,5cyclictest2773043-21aten_r2power_po09:57:421
2705564993628,5cyclictest2723546-21idleruntime-cro08:10:001
2705564993628,5cyclictest0-21swapper/108:00:011
2705564993628,5cyclictest0-21swapper/108:00:011
2705564993626,6cyclictest421-21avahi-daemon12:08:381
270556399366,25cyclictest2835098-21ssh11:32:520
270556399364,24cyclictest2805598-21ssh10:47:560
2705563993630,4cyclictest2741534-21diskmemload12:20:020
2705563993628,5cyclictest2831447-21cpu11:27:470
2705563993627,6cyclictest2841092-21/usr/sbin/munin11:42:500
2705563993626,8cyclictest2802586-21sshd10:42:550
2705563993626,7cyclictest2793211-21timerandwakeup10:28:070
2705563993626,7cyclictest2746267-21aten_r2power_en09:17:420
2705563993626,7cyclictest2746267-21aten_r2power_en09:17:420
2705563993625,6cyclictest0-21swapper/007:27:520
2705563993624,9cyclictest2726494-21head08:18:090
2705563993619,10cyclictest151ktimers/011:13:010
2705565993527,6cyclictest0-21swapper/207:25:012
2705565993527,6cyclictest0-21swapper/207:25:012
2705565993527,3cyclictest0-21swapper/212:00:462
2705565993526,6cyclictest2884902-21systemd-run12:40:022
2705565993526,5cyclictest2841873-21sshd11:42:542
270556599352,28cyclictest2792804-21ssh10:27:572
2705564993527,5cyclictest955-21mta-sts-daemon11:44:201
270556499352,26cyclictest2783399-21ssh10:12:561
270556499352,26cyclictest2783399-21ssh10:12:561
270556499351,8cyclictest2741534-21diskmemload11:12:561
270556399358,14cyclictest437-21mta-sts-daemon12:12:590
2705563993528,3cyclictest0-21swapper/012:40:010
2705563993526,7cyclictest491-21gdbus10:40:010
2705563993522,10cyclictest2764038-21unixbench-2d09:43:070
2705565993428,3cyclictest0-21swapper/210:55:012
2705565993428,3cyclictest0-21swapper/210:55:012
2705565993427,4cyclictest2833370-21latency_hist11:30:022
2705565993427,4cyclictest2798354-21kworker/u8:11-xprtiod11:00:002
2705565993426,5cyclictest2783019-21if_eth010:12:472
2705565993426,5cyclictest2783019-21if_eth010:12:472
2705565993426,5cyclictest2739078-21fschecks_time09:02:472
2705565993425,5cyclictest2875474-21sh12:27:592
2705564993429,2cyclictest0-21swapper/109:20:011
2705564993429,2cyclictest0-21swapper/109:20:011
2705564993427,5cyclictest2705562-21cyclictest11:48:081
2705564993427,5cyclictest2705562-21cyclictest11:48:081
2705564993427,4cyclictest2803818-21latency_hist10:45:031
2705564993427,4cyclictest2740038-21debian-sa109:05:011
2705564993426,4cyclictest2771879-21kworker/u8:8-xprtiod10:03:211
2705564993425,6cyclictest2875123-21tr12:27:511
2705564993425,6cyclictest2717561-21systemd-run07:50:021
270556399342,26cyclictest2756956-21ssh09:33:070
2705565993326,3cyclictest0-21swapper/211:32:542
270556599332,29cyclictest2776938-21ssh10:03:022
270556599332,24cyclictest2825229-21ssh11:18:072
270556499339,5cyclictest171rcu_preempt08:28:021
2705564993326,5cyclictest0-21swapper/108:15:021
2705564993324,5cyclictest2768256-21ssh09:50:011
2705564993322,8cyclictest2709445-21sed07:22:541
2705564993322,8cyclictest2709445-21sed07:22:541
270556499332,25cyclictest2743777-21ssh09:12:521
270556399333,26cyclictest591-21sshd09:57:540
2705563993325,5cyclictest2775335-21sh10:00:330
2705563993325,5cyclictest2749510-21/usr/sbin/munin09:22:450
2705563993324,6cyclictest2826538-21kworker/u8:2+xprtiod12:07:480
2705563993324,6cyclictest1910206-21cupsd10:56:510
2705563993323,6cyclictest12450irq/32-eth010:22:570
2705563993322,8cyclictest0-21swapper/007:55:010
2705563993322,8cyclictest0-21swapper/007:55:010
2705565993224,5cyclictest2761533-21latency_hist09:40:022
2705565993224,4cyclictest957-21mta-sts-daemon07:12:592
2705565993224,4cyclictest0-21swapper/209:47:582
270556499324,4cyclictest2756856-21ssh09:33:011
2705564993225,4cyclictest2726112-21sendmail_mailqu08:18:041
2705564993223,5cyclictest421-21avahi-daemon10:05:241
2705564993223,5cyclictest421-21avahi-daemon10:05:241
2705563993225,4cyclictest0-21swapper/009:12:420
2705563993224,5cyclictest2855737-21ntp_kernel_err12:02:520
2705563993221,8cyclictest2813902-21kworker/u8:2-rpciod11:07:560
2705563993218,12cyclictest2850111-21sh11:55:340
270556599316,17cyclictest437-21mta-sts-daemon10:07:552
270556599316,17cyclictest437-21mta-sts-daemon10:07:552
270556599313,21cyclictest657-21snmpd08:32:532
2705565993124,3cyclictest0-21swapper/208:00:012
2705565993124,3cyclictest0-21swapper/208:00:012
270556599312,26cyclictest2770372-21ssh09:53:012
2705565993122,6cyclictest2733983-21unixbench_singl08:43:082
2705565993122,5cyclictest2845678-21unixbench_singl11:48:092
2705565993122,5cyclictest2845678-21unixbench_singl11:48:092
270556599312,23cyclictest2863819-21rm12:13:042
2705564993126,2cyclictest0-21swapper/109:27:411
2705564993123,5cyclictest2712628-21sendmail_mailst07:32:591
2705564993122,6cyclictest2849672-21systemd-run11:55:011
2705564993122,6cyclictest2849672-21systemd-run11:55:011
270556499312,25cyclictest421-21avahi-daemon12:13:021
270556499312,24cyclictest2805655-21ssh10:47:581
2705564993121,6cyclictest266-21systemd-journal07:40:011
2705563993124,5cyclictest2725057-21systemd-run08:15:010
2705563993122,6cyclictest2737025-21munin-plugin-st08:55:020
2705563993122,6cyclictest0-21swapper/011:05:010
2705565993024,3cyclictest0-21swapper/209:57:442
2705565993024,3cyclictest0-21swapper/208:15:012
2705565993023,4cyclictest955-21mta-sts-daemon12:08:442
2705564993022,5cyclictest2730070-21fschecks_time08:32:461
2705564993022,5cyclictest2717849-21aten_r2power_en07:52:491
2705564993022,5cyclictest2717849-21aten_r2power_en07:52:491
270556499302,22cyclictest2818720-21ssh11:08:041
2705564993021,6cyclictest599-21lldpd11:22:471
2705564993021,6cyclictest599-21lldpd11:22:471
2705564993021,6cyclictest2738456-21unixbench_singl08:58:041
2705564993017,10cyclictest2710873-21needreboot07:27:491
2705563993022,5cyclictest2766597-21latency09:47:490
2705563993021,6cyclictest2871187-21kernelversion12:22:490
2705563993021,6cyclictest2769645-21aten_r2power_vo09:52:440
2705563993018,9cyclictest2707267-21kworker/u8:5-xprtiod07:33:130
2705565992922,4cyclictest2839026-21unixbench_singl11:38:072
2705565992922,4cyclictest2839026-21unixbench_singl11:38:072
2705565992922,4cyclictest2707267-21kworker/u8:5-xprtiod07:35:022
270556599291,20cyclictest2707983-21sshd07:17:552
270556599291,20cyclictest2707983-21sshd07:17:552
2705564992921,5cyclictest2751566-21munin-run09:25:011
2705564992921,5cyclictest1-21systemd08:25:021
2705564992918,8cyclictest2812640-21sh10:58:051
2705563992921,6cyclictest421-21avahi-daemon08:25:010
270556599285,14cyclictest437-21mta-sts-daemon07:42:582
2705565992823,2cyclictest0-21swapper/210:20:012
2705565992822,3cyclictest0-21swapper/209:10:012
2705565992813,8cyclictest2741534-21diskmemload11:52:562
2705565992813,8cyclictest2741534-21diskmemload11:52:562
2705564992822,3cyclictest0-21swapper/110:28:071
2705564992821,4cyclictest959-21mta-sts-daemon08:36:081
2705564992815,10cyclictest261rcuc/108:02:571
2705564992815,10cyclictest261rcuc/108:02:571
2705563992822,4cyclictest2705584-21latency_hist07:10:020
2705563992821,5cyclictest1910230-21cups-browsed08:08:420
2705563992821,4cyclictest422-21dbus-daemon09:05:010
2705563992821,4cyclictest2714541-21munin-run07:40:010
2705563992819,4cyclictest2705562-21cyclictest07:58:030
2705563992819,4cyclictest2705562-21cyclictest07:58:030
270556399281,20cyclictest2783337-21ssh10:12:540
270556399281,20cyclictest2783337-21ssh10:12:540
2705565992721,3cyclictest0-21swapper/208:25:012
2705565992720,5cyclictest959-21mta-sts-daemon08:08:002
2705565992719,5cyclictest2719033-21cron07:55:012
2705565992719,5cyclictest2719033-21cron07:55:012
2705564992721,3cyclictest0-21swapper/107:12:451
2705564992720,4cyclictest2734787-21missed_timers08:47:501
270556399274,11cyclictest2741534-21diskmemload11:52:040
270556399274,11cyclictest2741534-21diskmemload11:52:040
2705563992720,4cyclictest2735485-21head08:48:050
2705565992620,3cyclictest0-21swapper/207:47:542
2705563992621,3cyclictest2707573-21fschecks_count07:17:460
2705563992621,3cyclictest2707573-21fschecks_count07:17:460
2705563992620,4cyclictest1910230-21cups-browsed08:30:450
2705563992619,3cyclictest0-21swapper/009:07:480
2705563992618,6cyclictest2705562-21cyclictest07:48:030
270556399261,22cyclictest1-21systemd07:42:550
2705565992515,7cyclictest2729541-21/usr/sbin/munin08:30:022
2705563992519,3cyclictest0-21swapper/007:13:080
2705563992516,7cyclictest266-21systemd-journal07:20:550
2705563992516,7cyclictest266-21systemd-journal07:20:550
270556399251,20cyclictest2879422-21ntpq12:32:590
270556599243,13cyclictest437-21mta-sts-daemon08:57:562
270556499242,18cyclictest2715498-21sshd07:42:551
270556399243,5cyclictest171rcu_preempt08:02:560
270556399243,5cyclictest171rcu_preempt08:02:560
2705563992416,5cyclictest2734056-21systemd-run08:45:020
2705563992412,10cyclictest0-21swapper/008:28:100
2705564992317,4cyclictest2705562-21cyclictest08:52:521
2705565992216,3cyclictest0-21swapper/207:10:022
270556599221,16cyclictest2734954-21sshd08:47:552
2705564992115,3cyclictest2732820-21aten_r2power_cu08:42:421
270556699209,7cyclictest0-21swapper/311:18:023
270556699199,8cyclictest0-21swapper/310:47:543
270556699192,11cyclictest0-21swapper/309:17:563
270556699192,11cyclictest0-21swapper/309:17:563
270556699192,10cyclictest0-21swapper/307:22:543
270556699192,10cyclictest0-21swapper/307:22:543
2705566991811,4cyclictest0-21swapper/309:42:563
270556699181,11cyclictest0-21swapper/307:57:513
270556699181,11cyclictest0-21swapper/307:57:513
270556699179,6cyclictest0-21swapper/310:03:013
270556699173,8cyclictest0-21swapper/308:07:583
270556699172,9cyclictest0-21swapper/311:22:583
270556699172,9cyclictest0-21swapper/311:22:583
270556699172,10cyclictest0-21swapper/311:37:553
270556699172,10cyclictest0-21swapper/311:37:553
270556699171,11cyclictest0-21swapper/310:10:563
270556699171,11cyclictest0-21swapper/310:10:563
270556699168,5cyclictest0-21swapper/312:12:593
270556699161,11cyclictest0-21swapper/312:32:553
270556699161,10cyclictest0-21swapper/312:17:573
270556699161,10cyclictest0-21swapper/307:53:023
270556699161,10cyclictest0-21swapper/307:53:023
270556699160,11cyclictest0-21swapper/307:47:553
270556699158,5cyclictest0-21swapper/311:32:543
270556699157,5cyclictest0-21swapper/310:32:573
270556699153,8cyclictest0-21swapper/310:38:043
270556699151,11cyclictest0-21swapper/309:10:013
270556699151,10cyclictest0-21swapper/312:27:573
270556699151,10cyclictest0-21swapper/312:07:433
270556699151,10cyclictest0-21swapper/311:27:563
270556699151,10cyclictest0-21swapper/311:00:033
270556699151,10cyclictest0-21swapper/310:42:543
270556699151,10cyclictest0-21swapper/310:27:543
270556699151,10cyclictest0-21swapper/310:17:543
270556699151,10cyclictest0-21swapper/310:05:173
270556699151,10cyclictest0-21swapper/310:05:173
270556699151,10cyclictest0-21swapper/309:57:573
270556699151,10cyclictest0-21swapper/309:52:153
270556699151,10cyclictest0-21swapper/309:22:203
270556699151,10cyclictest0-21swapper/309:11:273
270556699151,10cyclictest0-21swapper/308:47:573
270556699151,10cyclictest0-21swapper/308:02:563
270556699151,10cyclictest0-21swapper/308:02:563
270556699151,10cyclictest0-21swapper/307:27:533
270556699150,11cyclictest0-21swapper/312:22:553
270556699150,11cyclictest0-21swapper/311:12:583
270556699150,10cyclictest0-21swapper/308:52:533
270556699150,10cyclictest0-21swapper/308:17:583
270556499151,4cyclictest171rcu_preempt07:10:011
270556699147,5cyclictest0-21swapper/309:27:533
270556699143,6cyclictest0-21swapper/310:22:533
270556699141,9cyclictest0-21swapper/311:55:143
270556699141,9cyclictest0-21swapper/310:50:123
270556699141,9cyclictest0-21swapper/310:50:123
270556699141,9cyclictest0-21swapper/309:37:463
270556699141,9cyclictest0-21swapper/307:37:543
270556699141,10cyclictest0-21swapper/311:53:083
270556699141,10cyclictest0-21swapper/311:53:083
270556699141,10cyclictest0-21swapper/311:43:043
270556699141,10cyclictest0-21swapper/311:05:023
270556699141,10cyclictest0-21swapper/309:32:593
270556699141,10cyclictest0-21swapper/308:57:513
270556699141,10cyclictest0-21swapper/308:37:513
270556699140,11cyclictest0-21swapper/307:10:013
270556699140,10cyclictest0-21swapper/312:37:583
270556699140,10cyclictest0-21swapper/312:01:013
270556699140,10cyclictest0-21swapper/311:06:353
270556699140,10cyclictest0-21swapper/308:23:003
270556699140,10cyclictest0-21swapper/308:13:103
270556699140,10cyclictest0-21swapper/307:12:563
270556699132,9cyclictest0-21swapper/309:47:553
270556699131,9cyclictest0-21swapper/309:02:463
270556699131,9cyclictest0-21swapper/308:32:473
270556699131,9cyclictest0-21swapper/308:27:523
270556699131,9cyclictest0-21swapper/307:42:413
270556699131,9cyclictest0-21swapper/307:32:483
270556699131,9cyclictest0-21swapper/307:16:483
270556699131,9cyclictest0-21swapper/307:16:483
270556699130,9cyclictest0-21swapper/308:41:543
270556699121,9cyclictest0-21swapper/311:47:513
270556699121,9cyclictest0-21swapper/311:47:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional