You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-01 - 04:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Tue Jul 01, 2025 00:41:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3370270995847,7cyclictest3496187-21unixbench-2d23:18:052
3370269995636,17cyclictest3434300-21idleruntime-cro21:50:011
3370269995241,7cyclictest3511115-21idleruntime-cro23:40:011
3370269995042,5cyclictest3551812-21ssh00:40:011
3370269995038,7cyclictest3507642-21cron23:35:011
337026999492,17cyclictest3547069-21ntpq00:32:531
3370270994840,5cyclictest422-21dbus-daemon21:20:012
3370268994836,9cyclictest3497110-21idleruntime-cro23:20:010
3370269994737,6cyclictest1-21systemd22:50:011
3370269994735,9cyclictest3538188-21ls00:20:011
337026999473,35cyclictest657-21snmpd00:27:541
3370269994725,18cyclictest0-21swapper/100:00:021
3370269994635,7cyclictest3408157-21fschecks_count21:12:471
3370269994629,14cyclictest421-21avahi-daemon23:08:351
3370269994623,19cyclictest0-21swapper/122:18:041
337026999462,16cyclictest3412187-21ssh21:18:011
337026999462,15cyclictest3386199-21ntpq20:02:531
3370270994537,5cyclictest3469125-21systemd-run22:40:012
3370269994531,9cyclictest1590858-21cups-browsed22:12:171
337026999452,6cyclictest3372671-21ntpq19:17:511
337026999451,11cyclictest3371168-21ntpq19:12:541
3370268994540,2cyclictest0-21swapper/022:12:470
3370268994527,13cyclictest12450irq/32-eth023:47:540
3370270994435,4cyclictest0-21swapper/223:47:542
3370270994434,6cyclictest421-21avahi-daemon23:41:182
337026999449,3cyclictest271ktimers/122:37:561
337026999443,6cyclictest3392144-21ntpq20:22:551
3370269994433,8cyclictest3397240-21sendmail20:40:011
337026999442,5cyclictest3395174-21ntpq20:32:561
337026999441,11cyclictest3384603-21ntpq19:57:511
3370270994334,6cyclictest3453612-21needreboot22:17:572
3370270994333,7cyclictest1-21systemd22:00:022
337026999435,29cyclictest657-21snmpd23:22:551
337026999433,6cyclictest3520159-21ssh23:52:571
3370269994335,5cyclictest3448205-21idleruntime-cro22:10:011
3370269994335,5cyclictest3448205-21idleruntime-cro22:10:011
337026999432,7cyclictest3374261-21ntpq19:23:021
337026999432,6cyclictest3513242-21ntpq23:43:021
3370269994317,23cyclictest3423080-21unixbench_multi21:33:091
3370268994336,4cyclictest3536624-21ntp_kernel_err00:17:510
337026999428,4cyclictest3441987-21ssh22:01:181
3370269994233,5cyclictest3465603-21sh22:35:001
3370269994230,7cyclictest0-21swapper/122:42:541
337026999422,7cyclictest3399650-21ntpq20:47:521
337026999421,11cyclictest3457276-21ntpq22:22:581
337026999421,11cyclictest3434722-21ssh21:50:331
337026999421,10cyclictest3483379-21ssh23:00:091
337026999421,10cyclictest3480743-21ssh22:57:101
337026999421,10cyclictest3424968-21ssh21:37:041
337026999421,10cyclictest3393702-21ntpq20:27:561
3370268994231,8cyclictest3525212-21cups-browsed00:37:410
3370270994130,8cyclictest3419266-21mailstats21:28:002
337026999417,4cyclictest3490857-21ssh23:11:181
337026999417,3cyclictest0-21swapper/120:17:551
3370269994133,5cyclictest3388219-21grep20:08:031
337026999412,8cyclictest3429433-21ssh21:42:521
337026999412,5cyclictest3404161-21ntpq21:02:541
337026999412,5cyclictest3402605-21ntpq20:57:511
337026999411,11cyclictest3414464-21ssh21:21:411
337026999411,10cyclictest3531521-21ssh00:10:341
337026999411,10cyclictest3531521-21ssh00:10:341
337026999411,10cyclictest3529465-21ssh00:06:141
337026999411,10cyclictest3438425-21ssh21:56:221
337026999411,10cyclictest3376760-21df19:32:451
3370270994032,5cyclictest3493638-21systemd23:15:022
3370270994031,6cyclictest1-21systemd21:05:022
3370270994030,6cyclictest3544365-21unixbench-2d00:28:062
3370269994030,6cyclictest3502756-21ntp_kernel_pll_23:27:571
337026999402,7cyclictest3539114-21sh00:21:411
337026999402,10cyclictest3398154-21ntpq20:42:531
337026999401,15cyclictest3419078-21ntp_kernel_pll_21:27:551
3370269994011,2cyclictest0-21swapper/123:47:561
337026999401,11cyclictest3478346-21ntpq22:53:031
337026999401,11cyclictest3383220-21ntpq19:52:561
337026999401,10cyclictest3527380-21ntpq00:02:561
337026999401,10cyclictest3459035-21ssh22:25:261
337026999401,10cyclictest3405527-21missed_timers21:07:511
337026899408,4cyclictest171rcu_preempt19:12:530
3370268994026,12cyclictest422-21dbus-daemon21:55:010
3370270993930,5cyclictest0-21swapper/221:42:552
3370270993928,8cyclictest3540877-21unixbench_multi00:23:072
337027099392,29cyclictest3471228-21ssh22:42:572
337026999390,10cyclictest3494176-21ssh23:15:541
3370268993934,2cyclictest0-21swapper/020:25:020
3370268993932,4cyclictest3423092-21unixbench_multi21:33:090
3370268993926,10cyclictest3389632-21unixbench_multi20:13:050
3370270993830,5cyclictest3379247-21munin-plugin-st19:40:012
3370270993830,3cyclictest0-21swapper/223:22:452
3370270993827,8cyclictest3415472-21missed_timers21:22:522
3370268993832,3cyclictest0-21swapper/022:18:050
3370268993831,4cyclictest3403405-21latency_hist21:00:030
337027099373,25cyclictest3408626-21ssh21:12:572
3370270993729,4cyclictest0-21swapper/223:38:092
3370270993727,3cyclictest0-21swapper/223:07:562
3370270993723,7cyclictest3375007-21/usr/sbin/munin19:27:452
337026999372,10cyclictest3381077-21tail19:47:411
3370268993730,5cyclictest421-21avahi-daemon19:53:010
3370268993730,4cyclictest3427440-21sendmail-msp21:40:010
3370268993722,9cyclictest3488661-21sh23:07:580
3370270993629,3cyclictest0-21swapper/220:27:522
3370270993628,5cyclictest3525156-21munin-plugin-st00:00:012
3370270993627,5cyclictest69-21kswapd022:58:012
3370270993626,4cyclictest0-21swapper/223:27:582
337026999361,14cyclictest3375795-21ps19:28:001
337026999361,12cyclictest3389212-21ntpq20:12:561
337026999361,11cyclictest3379281-21sendmail-msp19:40:011
3370268993630,4cyclictest3476174-21idleruntime-cro22:50:010
3370268993629,5cyclictest3408165-21fschecks_count21:12:470
3370268993624,3cyclictest0-21swapper/000:27:510
337026899362,29cyclictest422-21dbus-daemon00:02:200
3370270993527,5cyclictest3427404-21ssh21:40:012
3370270993526,5cyclictest3465610-21cut22:35:002
337027099352,25cyclictest3520144-21ssh23:52:572
337026999355,6cyclictest1590858-21cups-browsed19:41:371
337026999351,9cyclictest3401147-21ntpq20:52:521
3370268993528,4cyclictest3419213-21proc_pri21:28:000
3370270993426,5cyclictest3447350-21grep22:08:082
3370270993426,5cyclictest3447350-21grep22:08:082
3370270993426,5cyclictest3444828-21latency_hist22:05:022
3370270993425,5cyclictest3383642-21unixbench_multi19:53:042
3370270993424,8cyclictest3506816-21unixbench_singl23:33:062
3370268993427,4cyclictest3411542-21cpu21:17:440
3370268993426,5cyclictest3513749-21unixbench_singl23:43:130
3370268993416,4cyclictest3370267-21cyclictest22:07:090
3370268993416,4cyclictest3370267-21cyclictest22:07:090
337027099337,3cyclictest171rcu_preempt23:03:032
3370270993325,5cyclictest498-21wpa_supplicant00:17:482
3370270993325,4cyclictest3476295-21ssh22:50:032
3370270993324,5cyclictest0-21swapper/200:37:512
3370270993322,8cyclictest3405635-21ntp_kernel_pll_21:07:552
3370270993315,15cyclictest351ktimers/221:32:592
3370269993326,3cyclictest0-21swapper/119:10:011
3370268993327,4cyclictest3385809-21fschecks_time20:02:450
3370268993321,5cyclictest955-21mta-sts-daemon22:02:440
3370268993312,4cyclictest171rcu_preempt22:52:580
3370270993226,3cyclictest0-21swapper/200:35:032
3370270993225,4cyclictest3403242-21sendmail21:00:012
3370270993225,3cyclictest0-21swapper/200:12:502
3370270993225,3cyclictest0-21swapper/200:12:502
3370270993224,5cyclictest3384292-21fschecks_time19:57:462
337026899326,3cyclictest171rcu_preempt19:23:020
337026899322,24cyclictest3464117-21ssh22:32:530
3370268993220,9cyclictest3507649-21grep23:35:010
3370268993217,2cyclictest171rcu_preempt20:27:580
3370268993216,13cyclictest191rcuc/023:12:530
3370268993210,19cyclictest151ktimers/023:55:020
3370270993126,3cyclictest0-21swapper/222:30:022
3370270993125,3cyclictest0-21swapper/219:15:012
3370270993124,5cyclictest3450157-21/usr/sbin/munin22:12:552
3370270993124,4cyclictest3389823-21latency_hist20:15:022
337027099312,21cyclictest3457246-21ssh22:22:572
337026899315,10cyclictest171rcu_preempt20:42:500
3370268993124,5cyclictest3541772-21kworker/u8:6-rpciod00:25:010
3370268993124,4cyclictest3429152-21if_wlan021:42:470
3370268993123,6cyclictest3377480-21xargs19:33:040
3370268993123,5cyclictest3532719-21gltestperf00:12:420
3370268993123,5cyclictest3532719-21gltestperf00:12:420
3370268993122,6cyclictest3522139-21sshd23:55:500
337026899312,24cyclictest3509642-21ssh23:37:540
3370268993121,3cyclictest171rcu_preempt22:25:420
3370270993025,2cyclictest0-21swapper/220:40:012
3370270993023,4cyclictest3476189-21kworker/u8:8-xprtiod22:55:012
3370270993021,6cyclictest3386768-21munin-plugin-st20:05:012
3370270993021,6cyclictest3374792-21unin-run19:25:022
3370268993024,4cyclictest957-21mta-sts-daemon22:41:490
3370268993024,4cyclictest3389769-21kworker/u8:7-rpciod20:20:010
3370268993023,5cyclictest474-21in:imuxsock19:50:020
3370268993023,4cyclictest1-21systemd19:45:010
3370268993022,5cyclictest3406242-21cron21:10:010
3370268993016,10cyclictest0-21swapper/023:27:570
3370268993014,8cyclictest171rcu_preempt23:22:190
3370270992924,3cyclictest3379409-21kworker/u8:4+rpciod21:55:012
3370270992923,3cyclictest0-21swapper/219:50:012
3370270992922,4cyclictest956-21mta-sts-daemon00:02:202
3370270992921,6cyclictest657-21snmpd19:17:522
3370270992919,8cyclictest3401100-21ntp_kernel_err20:52:502
337027099291,12cyclictest0-21swapper/219:32:552
3370268992924,2cyclictest0-21swapper/021:21:180
3370268992922,4cyclictest422-21dbus-daemon21:55:090
3370268992921,6cyclictest3401732-21cron20:55:010
3370268992920,7cyclictest3433083-21proc_pri21:48:090
3370268992920,5cyclictest151ktimers/023:03:000
3370268992919,3cyclictest171rcu_preempt22:57:540
3370268992914,2cyclictest171rcu_preempt22:39:060
3370268992910,12cyclictest171rcu_preempt20:37:590
3370270992820,6cyclictest422-21dbus-daemon00:06:162
3370268992822,4cyclictest3395629-21tail20:33:050
3370268992821,3cyclictest0-21swapper/000:32:440
3370268992820,6cyclictest3385245-21sendmail20:00:010
3370268992819,7cyclictest3373148-21unixbench_multi19:18:010
3370268992818,6cyclictest12450irq/32-eth022:22:570
3370268992817,9cyclictest0-21swapper/000:10:010
3370270992719,5cyclictest3432210-21aten_r2power_cu21:47:472
3370270992719,5cyclictest1590858-21cups-browsed20:31:432
3370270992714,3cyclictest171rcu_preempt20:22:462
337026899273,19cyclictest1590858-21cups-browsed20:47:520
3370268992717,3cyclictest171rcu_preempt19:40:010
337026899271,20cyclictest657-21snmpd20:07:570
3370270992618,6cyclictest3380423-21switchtime19:43:072
3370270992519,3cyclictest0-21swapper/220:20:012
3370270992517,5cyclictest3388122-21unixbench-2d20:08:032
3370270992418,3cyclictest956-21mta-sts-daemon20:50:032
337026899243,5cyclictest14-21ksoftirqd/019:27:510
3370268992419,2cyclictest0-21swapper/019:10:010
3370268992415,2cyclictest171rcu_preempt21:02:560
3370270992210,9cyclictest3397765-21df_inode20:42:452
337027199214,11cyclictest0-21swapper/322:23:003
337027199194,7cyclictest0-21swapper/321:48:023
337027199194,7cyclictest0-21swapper/321:22:583
337027199181,9cyclictest0-21swapper/321:37:593
337027199174,6cyclictest0-21swapper/300:37:523
337027199174,6cyclictest0-21swapper/300:02:563
337027199174,4cyclictest0-21swapper/322:57:543
337027199172,8cyclictest0-21swapper/322:13:033
337027199164,4cyclictest0-21swapper/319:52:543
337027199162,8cyclictest0-21swapper/322:02:593
337027199162,7cyclictest0-21swapper/322:38:023
337027199161,7cyclictest0-21swapper/320:07:573
337027199161,6cyclictest0-21swapper/320:52:503
337027099167,6cyclictest453-21systemd-logind19:10:022
337027199155,5cyclictest0-21swapper/323:22:553
337027199154,6cyclictest0-21swapper/323:33:003
337027199154,6cyclictest0-21swapper/321:27:563
337027199153,7cyclictest0-21swapper/322:42:543
337027199153,6cyclictest0-21swapper/323:07:483
337027199152,8cyclictest0-21swapper/323:13:003
337027199152,7cyclictest0-21swapper/321:17:573
337027199152,7cyclictest0-21swapper/321:08:003
337027199152,6cyclictest0-21swapper/323:42:573
337027199152,6cyclictest0-21swapper/320:12:543
337027199152,6cyclictest0-21swapper/300:12:473
337027199152,6cyclictest0-21swapper/300:12:473
337027199151,8cyclictest0-21swapper/321:52:593
337027199151,8cyclictest0-21swapper/321:32:583
337027199151,7cyclictest0-21swapper/300:17:423
337027199151,5cyclictest0-21swapper/319:22:573
337027199143,7cyclictest0-21swapper/322:47:543
337027199143,6cyclictest0-21swapper/323:52:563
337027199142,7cyclictest0-21swapper/323:38:023
337027199142,7cyclictest0-21swapper/323:02:543
337027199142,7cyclictest0-21swapper/300:22:573
337027199142,6cyclictest0-21swapper/322:07:443
337027199142,6cyclictest0-21swapper/322:07:443
337027199141,7cyclictest0-21swapper/321:42:523
337027199141,7cyclictest0-21swapper/300:27:543
337027199141,6cyclictest0-21swapper/320:42:573
337027199141,6cyclictest0-21swapper/320:32:553
337027199141,6cyclictest0-21swapper/319:42:593
337027199137,4cyclictest0-21swapper/323:17:453
337027199134,6cyclictest0-21swapper/321:57:513
337027199134,4cyclictest0-21swapper/319:57:583
337027199133,6cyclictest0-21swapper/322:18:013
337027199133,5cyclictest0-21swapper/321:12:543
337027199133,5cyclictest0-21swapper/320:22:553
337027199132,7cyclictest0-21swapper/320:27:523
337027199132,6cyclictest0-21swapper/323:57:533
337027199132,6cyclictest0-21swapper/322:27:573
337027199132,6cyclictest0-21swapper/319:47:583
337027199132,6cyclictest0-21swapper/319:20:013
337027199132,6cyclictest0-21swapper/300:32:433
337027199131,5cyclictest0-21swapper/319:12:523
337027199126,4cyclictest0-21swapper/322:32:453
337027199125,5cyclictest0-21swapper/323:45:493
337027199122,7cyclictest0-21swapper/323:27:433
337027199122,5cyclictest0-21swapper/300:05:523
337027199121,6cyclictest0-21swapper/322:52:573
337027199121,6cyclictest0-21swapper/320:57:573
337027199121,5cyclictest0-21swapper/321:02:503
337027199112,6cyclictest0-21swapper/319:35:013
337027199111,6cyclictest0-21swapper/320:20:013
337027199105,3cyclictest0-21swapper/319:30:013
337027199101,5cyclictest0-21swapper/320:02:533
33702719992,5cyclictest0-21swapper/320:38:003
33702719992,5cyclictest0-21swapper/319:40:013
33702719984,3cyclictest0-21swapper/320:45:033
33702719980,5cyclictest431ktimers/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional