You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 07:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Tue Dec 02, 2025 00:39:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1497372995446,5cyclictest1-21systemd22:35:010
1497373995341,5cyclictest0-21swapper/100:19:191
1497374995041,5cyclictest1-21systemd23:00:022
1497373995043,3cyclictest0-21swapper/122:40:011
1497374994940,6cyclictest1680473-21systemd-run00:25:022
1497374994937,8cyclictest1-21systemd00:15:022
1497373994940,6cyclictest1541106-21systemd-run21:20:011
1497374994838,6cyclictest1643288-21cron23:35:012
1497374994819,9cyclictest1608356-21ntpq22:49:202
1497373994535,6cyclictest421-21avahi-daemon23:53:131
1497374994436,5cyclictest1578835-21latency_hist22:10:022
1497374994436,5cyclictest1578835-21latency_hist22:10:022
1497374994418,10cyclictest1545385-21ssh21:25:502
149737499441,35cyclictest1600745-21ssh22:39:232
1497373994436,5cyclictest1628207-21munin-plugin-st23:15:001
1497373994436,5cyclictest1628207-21munin-plugin-st23:15:001
1497374994335,5cyclictest1544810-21munin-plugin-st21:25:012
1497374994333,6cyclictest1623137-21fschecks_count23:09:172
1497374994318,9cyclictest1564788-21sh21:52:162
1497372994332,7cyclictest12450irq/32-eth022:09:180
1497372994332,7cyclictest12450irq/32-eth022:09:180
149737499423,32cyclictest1533350-21diskmemload23:14:232
149737499423,32cyclictest1533350-21diskmemload23:14:232
1497374994223,16cyclictest1586425-21ssh22:20:012
1497374994218,16cyclictest0-21swapper/220:24:202
1497374994218,16cyclictest0-21swapper/220:24:202
1497373994234,5cyclictest1620596-21munin-plugin-st23:05:011
1497372994235,4cyclictest1518355-21sendmail20:20:010
1497374994131,7cyclictest1684396-21sh00:31:142
1497374994129,9cyclictest1657406-21latency23:54:122
1497374994127,8cyclictest1646327-21ntpq23:39:222
149737299413,29cyclictest657-21snmpd23:59:240
1497374994031,5cyclictest1503384-21systemd-run19:30:012
1497374994030,6cyclictest0-21swapper/221:14:202
1497374994028,7cyclictest1532858-21ntpq21:09:232
1497374994027,8cyclictest1513276-21ntpq20:04:212
1497374994025,11cyclictest1561171-21ssh21:47:372
1497374994017,8cyclictest1610446-21diskmemload22:52:102
1497373994033,3cyclictest0-21swapper/100:09:201
1497372994033,5cyclictest1658568-21systemd-run23:55:010
1497374993932,4cyclictest1597795-21idleruntime-cro22:35:012
1497374993927,8cyclictest1510335-21ntpq19:54:222
1497374993926,8cyclictest1558880-21ntpq21:44:222
1497374993925,8cyclictest1504242-21ntpq19:34:182
1497374993925,8cyclictest1504242-21ntpq19:34:182
1497374993925,8cyclictest1498232-21ntpq19:14:162
149737499392,31cyclictest1593059-21sh22:29:172
1497374993923,11cyclictest1526833-21ntpq20:49:232
1497374993919,14cyclictest1540200-21ssh21:19:192
1497374993917,7cyclictest1587984-21sh22:22:542
1497374993916,14cyclictest0-21swapper/221:59:312
149737499391,33cyclictest1653980-21ssh23:49:232
1497373993933,3cyclictest0-21swapper/100:40:011
1497373993931,5cyclictest1592399-21aten_r2power_cu22:29:081
1497372993932,4cyclictest1690738-21systemd-run00:40:030
1497372993930,4cyclictest957-21mta-sts-daemon22:19:250
1497372993925,9cyclictest12450irq/32-eth023:09:260
1497374993830,5cyclictest1651033-21latency_hist23:45:022
1497374993828,7cyclictest1690695-21idleruntime-cro00:40:012
1497374993827,7cyclictest1581719-21ntpq22:14:202
1497374993826,7cyclictest1529717-21ntpq20:59:212
1497374993825,8cyclictest1661297-21ntpq23:59:222
1497374993824,9cyclictest1514781-21ntpq20:09:232
1497374993818,4cyclictest1631853-21sh23:19:502
1497374993817,9cyclictest1633635-21sh23:23:042
1497374993816,7cyclictest1619737-21awk23:04:252
1497374993816,6cyclictest1549396-21ssh21:31:292
1497374993815,9cyclictest1667857-21ssh00:08:542
1497373993830,5cyclictest1581545-21needreboot22:14:151
1497373993829,6cyclictest498-21wpa_supplicant00:04:211
1497373993829,6cyclictest1524381-21systemd-run20:40:011
1497373993829,6cyclictest0-21swapper/123:30:021
149737399381,29cyclictest1642499-21ssh23:34:231
149737299383,27cyclictest1589352-21ssh22:24:210
1497372993831,4cyclictest1512377-21systemd-run20:00:010
1497372993825,11cyclictest1548500-21ssh21:30:010
1497372993824,11cyclictest421-21avahi-daemon23:36:090
149737299381,30cyclictest1533350-21diskmemload00:04:250
1497374993727,7cyclictest1517371-21df_inode20:19:102
1497374993724,8cyclictest1508755-21ntpq19:49:172
1497374993724,10cyclictest1499719-21/usr/sbin/munin19:19:162
1497374993723,9cyclictest1638754-21ntpq23:29:242
1497374993723,9cyclictest1507308-21ntpq19:44:202
1497374993719,12cyclictest1525365-21ntpq20:44:242
1497374993712,10cyclictest1674145-21diskmemload00:16:422
1497373993729,4cyclictest0-21swapper/100:34:151
1497373993728,6cyclictest422-21dbus-daemon19:20:011
1497373993727,8cyclictest1631936-21sh23:20:011
1497372993731,2cyclictest0-21swapper/020:35:010
1497372993729,5cyclictest1627927-21unixbench_singl23:14:300
1497372993729,5cyclictest1627927-21unixbench_singl23:14:300
1497372993726,3cyclictest0-21swapper/023:34:310
1497372993724,10cyclictest1616816-21cron23:00:010
1497374993626,7cyclictest1523751-21ntpq20:39:192
1497374993625,7cyclictest1574067-21ntpq22:04:192
1497374993625,7cyclictest1574067-21ntpq22:04:192
1497374993625,7cyclictest1531284-21ntpq21:04:202
1497374993624,9cyclictest1682855-21ntpq00:29:182
1497374993624,9cyclictest1552896-21ssh21:35:592
1497374993623,10cyclictest1522303-21/usr/sbin/munin20:34:262
1497374993621,9cyclictest1516328-21ntpq20:14:262
1497374993621,10cyclictest1528390-21ps20:54:242
1497373993630,3cyclictest0-21swapper/123:40:011
1497373993628,5cyclictest1672862-21unixbench_multi00:14:321
1497373993627,5cyclictest422-21dbus-daemon19:45:021
1497373993627,5cyclictest1538462-21kworker/u8:6-xprtiod21:29:271
149737399361,30cyclictest1562734-21ssh21:49:241
149737299363,24cyclictest1555109-21ssh21:39:200
1497374993525,6cyclictest1501313-21ntpq19:24:192
1497374993524,7cyclictest1520888-21open_inodes20:29:252
1497374993524,7cyclictest1505082-21sshd19:36:072
1497374993523,8cyclictest1511708-21ntp_kernel_err19:59:172
1497374993522,8cyclictest1662602-21ssh00:00:192
1497374993521,11cyclictest1602633-21ssh22:41:432
1497373993529,3cyclictest0-21swapper/122:04:301
1497373993529,3cyclictest0-21swapper/122:04:301
1497373993526,5cyclictest421-21avahi-daemon23:22:121
1497373993526,3cyclictest0-21swapper/122:59:201
1497372993528,4cyclictest955-21mta-sts-daemon00:34:150
1497372993527,6cyclictest3827321-21cups-browsed23:49:560
1497372993526,7cyclictest1507857-21sh19:45:010
1497373993428,3cyclictest0-21swapper/100:00:031
1497373993427,4cyclictest1577786-21sed22:09:151
1497373993427,4cyclictest1577786-21sed22:09:151
1497372993428,4cyclictest1516763-21unixbench_multi20:14:330
1497372993426,5cyclictest151ktimers/019:29:180
1497372993424,5cyclictest0-21swapper/022:54:210
1497372993412,16cyclictest12450irq/32-eth021:44:260
1497373993328,2cyclictest0-21swapper/123:45:011
1497373993327,3cyclictest0-21swapper/121:55:021
1497373993327,3cyclictest0-21swapper/121:34:281
1497373993327,3cyclictest0-21swapper/121:05:011
1497373993326,4cyclictest1612521-21awk22:54:281
1497373993324,6cyclictest3827318-21cupsd21:59:431
1497373993324,6cyclictest1537331-21cron21:15:011
1497373993323,7cyclictest1516243-21ntp_kernel_pll_20:14:221
1497372993327,3cyclictest0-21swapper/021:59:330
1497372993324,7cyclictest1501770-21unixbench_multi19:24:290
1497372993323,8cyclictest266-21systemd-journal21:45:090
149737299332,24cyclictest1668734-21ssh00:09:210
1497373993225,3cyclictest0-21swapper/121:39:201
1497373993224,5cyclictest1589931-21unixbench_singl22:24:311
1497373993224,5cyclictest1512333-21cron20:00:011
1497373993223,7cyclictest1514751-21ntp_kernel_pll_20:09:221
1497373993223,6cyclictest3827318-21cupsd21:21:211
1497373993223,4cyclictest0-21swapper/119:14:191
1497373993212,16cyclictest271ktimers/123:07:281
149737299329,12cyclictest12450irq/32-eth021:19:200
1497372993226,4cyclictest698-21cron20:45:010
1497372993225,4cyclictest1534231-21sh21:11:160
1497372993225,4cyclictest1510836-21cron19:55:010
149737299322,15cyclictest1551307-21ssh21:34:170
1497372993218,5cyclictest0-21swapper/023:29:100
1497373993125,3cyclictest0-21swapper/122:44:241
1497373993124,3cyclictest0-21swapper/122:19:241
1497373993124,3cyclictest0-21swapper/100:29:201
1497373993123,5cyclictest453-21systemd-logind23:49:251
1497373993123,5cyclictest421-21avahi-daemon22:32:361
1497373993122,6cyclictest1518920-21fschecks_time20:24:121
1497373993122,6cyclictest1518920-21fschecks_time20:24:121
1497373993120,8cyclictest1678846-21/usr/sbin/munin00:24:271
149737299318,17cyclictest657-21snmpd20:54:160
149737299312,25cyclictest1600718-21ssh22:39:220
1497372993121,7cyclictest1505718-21ntp_kernel_err19:39:160
1497372993121,6cyclictest3827321-21cups-browsed22:49:220
149737299311,22cyclictest1604454-21ssh22:44:170
1497373993023,4cyclictest956-21mta-sts-daemon22:49:121
1497373993023,3cyclictest0-21swapper/119:39:111
1497373993022,3cyclictest0-21swapper/121:44:191
1497372993024,4cyclictest1533350-21diskmemload23:05:020
1497372993024,4cyclictest1504876-21systemd-run19:35:010
1497372993024,4cyclictest1504876-21systemd-run19:35:010
1497372993024,3cyclictest0-21swapper/023:19:270
149737299302,23cyclictest1650153-21ssh23:44:210
1497372993021,7cyclictest1500408-21latency_hist19:20:020
1497372993017,10cyclictest266-21systemd-journal21:22:010
1497372993016,3cyclictest171rcu_preempt00:24:240
1497373992922,4cyclictest959-21mta-sts-daemon20:59:131
1497372992923,4cyclictest421-21avahi-daemon20:39:260
149737299292,21cyclictest1634906-21ssh23:24:220
1497372992920,7cyclictest1573491-21cpu22:04:080
1497372992920,7cyclictest1573491-21cpu22:04:080
1497372992920,4cyclictest1593062-21ssh22:29:180
149737299291,21cyclictest1671881-21ssh00:14:120
1497374992818,6cyclictest1497377-21systemd-run19:10:012
1497373992821,3cyclictest0-21swapper/121:10:011
149737299285,15cyclictest437-21mta-sts-daemon00:19:200
1497372992820,5cyclictest3827321-21cups-browsed22:11:520
1497372992817,8cyclictest1514691-21needreboot20:09:190
1497373992721,3cyclictest0-21swapper/119:25:011
1497373992719,4cyclictest0-21swapper/120:35:011
1497373992718,6cyclictest956-21mta-sts-daemon19:54:161
149737399271,16cyclictest1-21systemd19:10:021
149737299276,11cyclictest12450irq/32-eth021:04:180
149737299272,20cyclictest1681554-21ssh00:27:140
1497372992718,7cyclictest1526776-21ntp_kernel_pll_20:49:210
1497372992716,8cyclictest0-21swapper/021:10:010
1497373992620,3cyclictest0-21swapper/120:30:001
1497372992619,4cyclictest1566330-21ntp_kernel_err21:54:170
1497372992617,5cyclictest1521354-21latency_hist20:30:000
1497373992519,3cyclictest959-21mta-sts-daemon20:49:301
1497373992519,3cyclictest0-21swapper/119:35:021
1497373992519,3cyclictest0-21swapper/119:35:021
1497373992519,2cyclictest0-21swapper/120:05:011
1497373992518,4cyclictest1509221-21unixbench-2d19:49:281
1497373992518,4cyclictest1502413-21fschecks_count19:29:121
1497373992517,5cyclictest1527892-21fschecks_count20:54:111
1497372992521,2cyclictest0-21swapper/020:24:140
1497372992521,2cyclictest0-21swapper/020:24:140
1497373992413,7cyclictest1518362-21idleruntime-cro20:20:011
1497372992418,4cyclictest1486281-21kworker/u8:3+xprtiod19:10:010
1497372992418,4cyclictest0-21swapper/019:15:010
1497372992412,9cyclictest0-21swapper/020:05:020
1497372992317,4cyclictest3827321-21cups-browsed19:48:420
1497373992215,4cyclictest1524905-21fschecks_count20:44:111
1497372992216,4cyclictest591-21sshd20:55:090
149737599193,8cyclictest0-21swapper/322:29:243
149737599183,9cyclictest0-21swapper/321:14:203
149737599182,8cyclictest0-21swapper/322:14:213
149737599172,9cyclictest0-21swapper/323:04:213
149737599172,8cyclictest0-21swapper/323:09:123
149737599172,7cyclictest0-21swapper/322:04:063
149737599172,7cyclictest0-21swapper/322:04:063
149737599171,8cyclictest0-21swapper/321:44:263
149737599163,7cyclictest0-21swapper/322:49:183
149737599162,9cyclictest0-21swapper/322:34:113
149737599162,8cyclictest0-21swapper/323:59:303
149737599162,8cyclictest0-21swapper/323:29:183
149737599162,8cyclictest0-21swapper/322:59:203
149737599162,8cyclictest0-21swapper/300:04:193
149737599161,9cyclictest0-21swapper/322:44:193
149737599161,8cyclictest0-21swapper/300:14:203
149737599161,8cyclictest0-21swapper/300:09:193
149737599152,8cyclictest0-21swapper/323:24:093
149737599152,8cyclictest0-21swapper/322:54:193
149737599152,8cyclictest0-21swapper/300:24:183
149737599152,8cyclictest0-21swapper/300:19:173
149737599152,7cyclictest0-21swapper/321:49:183
149737599152,7cyclictest0-21swapper/321:19:163
149737599152,6cyclictest0-21swapper/320:54:223
149737599152,6cyclictest0-21swapper/320:09:213
149737599151,9cyclictest0-21swapper/321:34:183
149737599151,8cyclictest0-21swapper/323:34:243
149737599151,8cyclictest0-21swapper/323:19:123
149737599151,8cyclictest0-21swapper/322:19:313
149737599151,8cyclictest0-21swapper/322:09:143
149737599151,8cyclictest0-21swapper/322:09:143
149737599151,7cyclictest0-21swapper/323:44:213
149737599151,7cyclictest0-21swapper/321:04:243
149737599151,7cyclictest0-21swapper/320:14:243
149737599151,6cyclictest0-21swapper/319:44:163
149737599151,10cyclictest0-21swapper/322:24:203
149737599142,7cyclictest0-21swapper/323:39:193
149737599142,7cyclictest0-21swapper/300:29:193
149737599142,6cyclictest0-21swapper/300:34:053
149737599141,9cyclictest0-21swapper/323:49:143
149737599141,9cyclictest0-21swapper/323:14:063
149737599141,9cyclictest0-21swapper/323:14:063
149737599141,9cyclictest0-21swapper/321:29:113
149737599141,8cyclictest0-21swapper/321:59:223
149737599141,8cyclictest0-21swapper/321:39:083
149737599141,8cyclictest0-21swapper/321:24:123
149737599141,6cyclictest0-21swapper/319:59:193
149737599141,6cyclictest0-21swapper/319:14:163
149737599134,6cyclictest0-21swapper/323:54:193
149737599132,7cyclictest0-21swapper/321:10:013
149737599132,7cyclictest0-21swapper/319:24:173
149737599132,5cyclictest0-21swapper/320:19:203
149737599131,8cyclictest0-21swapper/321:54:093
149737599131,7cyclictest0-21swapper/300:39:113
149737599131,6cyclictest0-21swapper/320:49:233
149737599131,6cyclictest0-21swapper/320:39:093
149737599131,6cyclictest0-21swapper/319:54:203
149737599131,6cyclictest0-21swapper/319:39:183
149737599131,6cyclictest0-21swapper/319:29:213
149737599122,6cyclictest0-21swapper/320:44:183
149737599122,5cyclictest0-21swapper/321:00:013
149737599121,7cyclictest0-21swapper/319:49:083
149737599121,6cyclictest0-21swapper/322:37:093
149737599121,6cyclictest0-21swapper/319:19:183
149737599112,5cyclictest0-21swapper/320:29:193
149737599111,6cyclictest0-21swapper/320:34:113
149737599111,5cyclictest0-21swapper/319:34:093
149737599111,5cyclictest0-21swapper/319:34:093
149737599103,5cyclictest0-21swapper/320:04:223
149737599101,6cyclictest0-21swapper/320:24:193
149737599101,6cyclictest0-21swapper/320:24:193
149737599101,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional