You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-13 - 07:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sun Jul 13, 2025 00:40:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1529849997465,6cyclictest151ktimers/023:12:590
1529849997465,6cyclictest151ktimers/023:12:590
1529849996437,4cyclictest12550irq/33-eth022:07:530
152985099623,47cyclictest1613040-21ssh22:18:041
152985099623,47cyclictest1613040-21ssh22:18:041
152984999614,46cyclictest1575251-21ssh21:23:000
1529851995951,5cyclictest0-21swapper/222:07:532
1529849995935,15cyclictest151ktimers/000:13:070
1529850995544,7cyclictest1685290-21cups-browsed00:13:001
1529851995443,7cyclictest3949213-21cups-browsed22:52:512
1529851995240,6cyclictest1550826-21grep20:20:012
1529850995038,7cyclictest1605589-21aten_r2power_fr22:07:521
152985099503,41cyclictest1704146-21ssh00:28:001
1529850994940,6cyclictest422-21dbus-daemon23:05:021
1529849994942,5cyclictest1582559-21/usr/sbin/munin21:33:110
1529851994740,4cyclictest955-21mta-sts-daemon00:38:002
1529851994739,5cyclictest1531347-21systemd-run19:15:012
1529851994739,5cyclictest1531347-21systemd-run19:15:012
152985099474,39cyclictest1690814-21ssh00:08:031
1529850994739,5cyclictest1699003-21ssh00:20:021
1529851994638,5cyclictest1-21systemd00:25:022
1529851994637,5cyclictest0-21swapper/222:00:012
1529850994637,4cyclictest0-21swapper/122:53:001
1529849994618,19cyclictest1690840-21ssh00:08:040
1529851994539,3cyclictest0-21swapper/223:48:082
1529850994539,3cyclictest0-21swapper/121:45:011
1529850994533,7cyclictest0-21swapper/122:03:051
152985099451,37cyclictest1687404-21ssh00:03:021
1529851994437,3cyclictest0-21swapper/200:15:012
1529851994435,5cyclictest0-21swapper/220:00:012
1529850994436,5cyclictest0-21swapper/123:28:061
1529850994335,5cyclictest1671229-21systemd-run23:40:021
1529850994335,5cyclictest1642398-21ssh23:00:011
1529850994332,5cyclictest0-21swapper/122:47:591
1529849994334,7cyclictest1532869-21sh19:20:010
152985199422,36cyclictest1680259-21ssh23:53:002
152985199422,36cyclictest1680259-21ssh23:53:002
1529850994232,7cyclictest1681638-21cron23:55:001
1529850994232,7cyclictest1681638-21cron23:55:001
152984999423,34cyclictest1602642-21ssh22:03:020
1529851994133,4cyclictest0-21swapper/200:00:012
1529851994132,5cyclictest0-21swapper/200:33:092
152985099414,32cyclictest1571626-21ssh21:17:581
1529849994134,5cyclictest1638168-21unixbench_singl22:53:140
1529849994133,5cyclictest1609168-21fschecks_time22:12:520
1529851994033,4cyclictest1532836-21munin-plugin-st19:20:012
1529851994032,5cyclictest1631094-21unixbench_multi22:43:112
1529851994032,5cyclictest1564353-21systemd-run21:05:012
1529851994029,6cyclictest1648255-21sh23:08:032
1529850994029,8cyclictest1575257-21ssh21:23:011
152985099402,6cyclictest171rcu_preempt20:13:021
152985099402,6cyclictest171rcu_preempt20:13:021
152985099401,35cyclictest1683824-21ssh23:58:021
1529851993933,3cyclictest0-21swapper/222:40:012
1529851993930,4cyclictest0-21swapper/222:23:072
1529851993929,6cyclictest1568268-21ssh21:13:032
1529851993929,5cyclictest0-21swapper/223:13:032
1529851993929,5cyclictest0-21swapper/223:13:032
1529850993931,5cyclictest1709547-21sh00:37:431
152985099392,28cyclictest1666395-21ssh23:33:021
152984999393,31cyclictest1659106-21ssh23:23:010
152985199383,28cyclictest1592415-21ssh21:48:052
1529851993831,4cyclictest0-21swapper/222:20:012
1529851993831,4cyclictest0-21swapper/222:20:012
1529851993829,6cyclictest422-21dbus-daemon22:30:012
1529851993829,5cyclictest1-21systemd23:20:012
1529851993827,8cyclictest1583360-21cut21:35:022
152985099383,29cyclictest3949213-21cups-browsed19:53:031
1529850993831,4cyclictest1550825-21munin-run20:20:011
1529850993830,5cyclictest1700524-21ssh00:22:551
1529849993832,4cyclictest0-21swapper/000:00:010
1529849993824,3cyclictest0-21swapper/022:38:010
152984999382,32cyclictest1655515-21ssh23:18:010
1529849993820,4cyclictest1586838-21latency_hist21:40:020
152984999381,29cyclictest698-21cron00:28:010
1529851993727,6cyclictest1669128-21aten2.4-expect23:37:512
1529851993726,8cyclictest1646048-21/usr/sbin/munin23:05:022
152985199372,32cyclictest1659239-21ssh23:23:062
1529851993721,13cyclictest1589043-21open_inodes21:43:092
152985199371,29cyclictest1673384-21ssh23:43:052
152985199371,29cyclictest1609665-21ssh22:13:042
1529850993730,4cyclictest0-21swapper/121:10:011
1529850993729,5cyclictest1586744-21sendmail21:40:011
152985099372,30cyclictest1623500-21ssh22:33:071
1529849993730,4cyclictest1600598-21latency_hist22:00:020
1529851993626,7cyclictest422-21dbus-daemon22:33:022
1529851993626,6cyclictest1586739-21grep21:40:002
1529850993628,5cyclictest1706744-21aten_r2power_vo00:32:531
1529850993628,5cyclictest1657003-21tr23:20:011
1529850993626,6cyclictest453-21systemd-logind19:35:011
1529849993628,6cyclictest0-21swapper/021:13:000
152984999362,27cyclictest1707281-21ssh00:33:020
1529849993622,12cyclictest1640733-21kernelversion22:57:570
1529851993527,5cyclictest1533384-21fschecks_count19:22:542
1529851993525,6cyclictest421-21avahi-daemon22:57:042
1529851993525,6cyclictest1685290-21cups-browsed00:05:372
1529850993527,5cyclictest477-21rs:main1
1529850993527,5cyclictest1617123-21unixbench_singl22:23:151
1529850993527,4cyclictest0-21swapper/123:42:551
1529850993526,6cyclictest1628322-21cron22:40:011
1529850993526,5cyclictest1540338-21munin-run19:45:011
1529850993525,8cyclictest1598654-21sh21:57:541
152985099352,27cyclictest1568356-21ssh21:13:061
152985099351,26cyclictest657-21snmpd19:38:021
152984999353,25cyclictest1687467-21ssh00:03:050
1529849993529,4cyclictest422-21dbus-daemon22:50:010
1529849993527,5cyclictest1702351-21latency_hist00:25:020
152985199347,17cyclictest1634118-21ssh22:48:002
1529851993428,3cyclictest0-21swapper/223:30:012
1529851993428,3cyclictest0-21swapper/221:20:012
1529851993427,3cyclictest0-21swapper/219:45:012
1529851993426,4cyclictest1638594-21kworker/u8:18-xprtiod23:35:012
1529851993419,3cyclictest171rcu_preempt00:18:282
1529850993426,5cyclictest1562851-21systemd-run21:00:021
1529850993426,3cyclictest0-21swapper/123:12:571
1529850993426,3cyclictest0-21swapper/123:12:571
1529850993425,5cyclictest0-21swapper/121:47:591
1529849993429,2cyclictest0-21swapper/000:20:010
1529849993428,3cyclictest1564337-21kworker/u8:3-xprtiod22:20:020
1529849993428,3cyclictest1564337-21kworker/u8:3-xprtiod22:20:020
152985199333,21cyclictest1704289-21ssh00:28:072
152985199332,8cyclictest1547210-21ntpq20:08:002
1529851993327,3cyclictest0-21swapper/220:05:022
1529851993323,7cyclictest1541743-21unixbench_multi19:48:112
1529850993325,5cyclictest1533390-21fschecks_time19:22:541
1529850993323,8cyclictest1553159-21needreboot20:27:571
1529850993321,8cyclictest0-21swapper/120:23:041
152984999333,4cyclictest171rcu_preempt19:52:590
1529849993328,3cyclictest0-21swapper/021:28:050
1529849993327,3cyclictest0-21swapper/021:53:020
1529849993326,4cyclictest1624787-21cron22:35:010
1529849993324,6cyclictest422-21dbus-daemon22:23:080
1529849993322,9cyclictest0-21swapper/021:17:520
1529849993321,7cyclictest69-21kswapd023:08:000
1529851993224,5cyclictest1535859-21systemd-run19:30:012
1529851993223,6cyclictest1559845-21systemd-run20:50:022
1529851993221,8cyclictest1577216-21ssh21:25:512
1529850993225,4cyclictest1677080-21mailstats23:48:111
1529849993224,6cyclictest1644918-21/usr/sbin/munin23:03:110
152985299312,11cyclictest0-21swapper/321:38:083
1529851993123,5cyclictest959-21mta-sts-daemon20:33:032
1529851993122,6cyclictest491-21gdbus21:22:512
1529851993119,9cyclictest1551753-21ntp_kernel_pll_20:23:012
1529851993115,10cyclictest957-21mta-sts-daemon20:43:022
1529850993124,5cyclictest1529848-21cyclictest20:03:121
1529850993124,3cyclictest0-21swapper/122:43:001
1529850993123,5cyclictest1555323-21debian-sa120:35:011
1529850993123,5cyclictest1529861-21unin-run19:10:011
152984999312,8cyclictest1676737-21sh23:48:010
1529849993125,4cyclictest0-21swapper/020:00:010
1529849993124,4cyclictest1549312-21cron20:15:010
1529849993124,4cyclictest1549312-21cron20:15:010
1529849993123,6cyclictest956-21mta-sts-daemon22:43:110
1529849993123,5cyclictest474-21in:imuxsock19:35:010
1529849993122,7cyclictest1666740-21wc23:33:130
1529849993121,8cyclictest1550153-21needreboot20:17:580
1529849993120,8cyclictest0-21swapper/021:48:000
1529849993115,3cyclictest171rcu_preempt23:43:050
152984999310,5cyclictest171rcu_preempt22:28:070
152985299301,9cyclictest0-21swapper/322:22:543
152985199309,5cyclictest36-21ksoftirqd/221:08:052
1529851993023,4cyclictest1515797-21kworker/u8:0-xprtiod19:40:012
1529851993020,5cyclictest0-21swapper/221:53:022
1529851993014,3cyclictest0-21swapper/219:53:022
1529850993023,4cyclictest0-21swapper/122:28:181
1529850993022,5cyclictest1546864-21fschecks_count20:07:541
1529850993020,4cyclictest959-21mta-sts-daemon23:23:111
152985099301,23cyclictest1581985-21ssh21:32:581
152984999303,22cyclictest1588931-21ssh21:43:040
1529849993025,3cyclictest151ktimers/020:48:030
1529849993024,4cyclictest955-21mta-sts-daemon23:55:010
1529849993024,4cyclictest955-21mta-sts-daemon23:55:010
1529849993023,5cyclictest3949213-21cups-browsed19:43:180
152984999302,24cyclictest1669838-21ssh23:38:020
1529849993021,6cyclictest1534346-21sh19:25:010
1529849993020,7cyclictest1564402-21latency_hist21:05:020
1529849993020,7cyclictest0-21swapper/020:25:000
1529849993020,7cyclictest0-21swapper/000:40:010
152985299292,9cyclictest0-21swapper/300:03:003
1529851992912,4cyclictest341rcuc/200:03:122
1529850992924,2cyclictest0-21swapper/122:12:571
1529850992920,6cyclictest422-21dbus-daemon21:55:011
152985099290,3cyclictest171rcu_preempt23:08:021
152984999294,6cyclictest171rcu_preempt19:38:070
152984999292,7cyclictest171rcu_preempt20:03:010
1529849992922,4cyclictest1565828-21cut21:10:010
1529849992921,5cyclictest3949213-21cups-browsed20:55:250
1529849992921,5cyclictest1553835-21munin-plugin-st20:30:010
1529849992921,4cyclictest0-21swapper/019:13:030
1529849992921,4cyclictest0-21swapper/019:13:030
1529849992920,7cyclictest1535796-21head19:28:100
1529849992920,6cyclictest0-21swapper/020:35:010
152985299283,8cyclictest0-21swapper/322:28:073
152985299282,9cyclictest0-21swapper/321:48:033
152985199288,4cyclictest36-21ksoftirqd/220:53:002
152985199286,5cyclictest36-21ksoftirqd/220:13:002
152985199286,5cyclictest36-21ksoftirqd/220:13:002
1529851992816,3cyclictest171rcu_preempt22:01:082
1529851992811,3cyclictest171rcu_preempt20:57:512
1529850992822,3cyclictest0-21swapper/121:28:111
1529850992822,3cyclictest0-21swapper/120:40:021
1529850992821,3cyclictest0-21swapper/120:00:011
1529849992822,4cyclictest1541751-21tail19:48:110
1529849992822,3cyclictest0-21swapper/023:28:010
1529851992720,4cyclictest351ktimers/220:35:042
1529851992718,6cyclictest421-21avahi-daemon19:32:512
1529850992721,3cyclictest0-21swapper/120:53:111
1529850992719,5cyclictest1558875-21fschecks_time20:47:541
152984999271,3cyclictest171rcu_preempt20:37:510
1529850992619,4cyclictest0-21swapper/119:12:591
1529850992619,4cyclictest0-21swapper/119:12:591
1529850992618,5cyclictest1564214-21tail21:03:121
1529850992618,5cyclictest1534624-21aten_r2power_cu19:27:481
1529850992617,6cyclictest491-21gdbus19:17:251
152985199255,5cyclictest36-21ksoftirqd/220:30:022
1529849992517,5cyclictest1560356-21fschecks_count20:52:530
152984999250,4cyclictest171rcu_preempt20:08:010
152985099241,17cyclictest3949213-21cups-browsed19:48:041
152984999241,19cyclictest14-21ksoftirqd/020:43:000
1529851992317,4cyclictest351ktimers/219:10:042
1529849992217,3cyclictest959-21mta-sts-daemon19:10:040
152985299203,10cyclictest0-21swapper/322:07:533
152985299203,10cyclictest0-21swapper/300:28:013
152985299202,8cyclictest0-21swapper/300:18:013
152985299201,8cyclictest0-21swapper/323:33:003
152985299191,10cyclictest0-21swapper/323:28:013
152985299184,9cyclictest0-21swapper/322:48:003
152985299184,10cyclictest0-21swapper/323:08:003
152985299183,8cyclictest0-21swapper/323:18:073
152985299183,6cyclictest0-21swapper/319:43:033
152985299182,9cyclictest0-21swapper/321:55:033
152985299182,9cyclictest0-21swapper/300:33:023
152985299182,9cyclictest0-21swapper/300:08:053
152985299182,10cyclictest0-21swapper/321:43:063
152985299181,9cyclictest0-21swapper/321:33:053
152985299181,10cyclictest0-21swapper/323:48:123
152985299174,8cyclictest0-21swapper/322:38:093
152985299173,8cyclictest0-21swapper/323:52:583
152985299173,8cyclictest0-21swapper/323:52:583
152985299172,9cyclictest0-21swapper/323:03:033
152985299172,9cyclictest0-21swapper/321:20:013
152985299163,6cyclictest0-21swapper/322:03:013
152985299162,8cyclictest0-21swapper/322:57:523
152985299162,8cyclictest0-21swapper/319:48:043
152985299162,6cyclictest0-21swapper/320:13:023
152985299162,6cyclictest0-21swapper/320:13:023
152985299162,10cyclictest0-21swapper/323:42:553
152985299161,9cyclictest0-21swapper/322:12:583
152985299161,8cyclictest0-21swapper/300:37:553
152985299161,7cyclictest0-21swapper/320:08:043
152985299161,7cyclictest0-21swapper/319:52:593
152985299161,7cyclictest0-21swapper/319:18:043
152985299161,6cyclictest0-21swapper/319:57:593
152985299152,9cyclictest0-21swapper/323:58:083
152985299152,9cyclictest0-21swapper/300:23:033
152985299152,8cyclictest0-21swapper/323:22:543
152985299152,8cyclictest0-21swapper/322:52:513
152985299152,8cyclictest0-21swapper/322:33:023
152985299152,8cyclictest0-21swapper/321:28:123
152985299152,8cyclictest0-21swapper/319:12:593
152985299152,8cyclictest0-21swapper/319:12:593
152985299152,7cyclictest0-21swapper/321:08:003
152985299152,6cyclictest0-21swapper/320:17:593
152985299152,5cyclictest0-21swapper/320:22:583
152985299151,9cyclictest0-21swapper/323:13:003
152985299151,9cyclictest0-21swapper/323:13:003
152985299151,9cyclictest0-21swapper/321:13:053
152985299151,8cyclictest0-21swapper/323:37:523
152985299151,8cyclictest0-21swapper/322:18:033
152985299151,8cyclictest0-21swapper/322:18:033
152985299151,7cyclictest0-21swapper/320:48:033
152985299151,7cyclictest0-21swapper/320:43:033
152985299151,7cyclictest0-21swapper/320:03:033
152985299144,7cyclictest0-21swapper/321:23:083
152985299144,4cyclictest0-21swapper/320:58:013
152985299141,8cyclictest0-21swapper/322:43:033
152985299141,6cyclictest0-21swapper/322:00:033
152985299136,4cyclictest0-21swapper/300:13:003
152985299134,5cyclictest0-21swapper/321:03:003
152985299131,6cyclictest0-21swapper/320:52:593
152985299131,6cyclictest0-21swapper/320:37:573
152985299131,6cyclictest0-21swapper/320:27:593
152985299124,5cyclictest0-21swapper/319:38:023
152985299123,5cyclictest0-21swapper/319:23:003
152985299122,6cyclictest0-21swapper/319:32:523
152985299122,5cyclictest0-21swapper/319:27:573
152985299111,6cyclictest0-21swapper/320:32:523
15298529992,4cyclictest0-21swapper/319:10:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional