You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 15:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Mon Nov 24, 2025 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1556750995445,6cyclictest299-21systemd-logind08:20:431
155674599500,3cyclictest0-21swapper/011:55:140
1556754994837,7cyclictest311ktimers/208:00:122
1556759994737,6cyclictest1623174-21idleruntime-cro10:20:003
155675499472,38cyclictest630-21snmpd11:45:132
1556750994736,7cyclictest0-21swapper/107:20:011
1556759994533,10cyclictest391ktimers/308:05:143
1556754994538,4cyclictest1556738-21cyclictest11:40:002
1556754994514,28cyclictest1629307-21sed10:35:252
155675499451,39cyclictest1631930-21apt-get10:45:142
155675499451,38cyclictest1602516-21apt-get09:20:122
1556750994536,6cyclictest1587074-21fschecks_count08:35:171
1556759994432,7cyclictest0-21swapper/312:36:233
155675499444,23cyclictest1584108-21ntp_states08:25:232
155675499441,40cyclictest1567757-21munin-node07:40:122
155675499441,25cyclictest1611822-21ntp_states09:45:242
155675499433,35cyclictest1664992-21/usr/sbin/munin12:20:182
155675499431,39cyclictest1630505-21/usr/sbin/munin10:40:182
155675499431,35cyclictest0-21swapper/209:00:122
155675499423,36cyclictest1586527-21cron08:33:012
155675499423,36cyclictest1578000-21cron08:08:012
1556754994231,7cyclictest1593510-21cron08:55:002
155675499422,37cyclictest1668522-21ntp_states12:30:202
155675499421,38cyclictest1626133-21/usr/sbin/munin10:25:262
155675499421,38cyclictest1607831-21/usr/sbin/munin09:35:172
155675499421,36cyclictest1615262-21/usr/sbin/munin09:55:222
155675499420,39cyclictest0-21swapper/208:15:122
155675499412,36cyclictest1601133-21/usr/sbin/munin09:15:192
155675499411,37cyclictest1648248-21/usr/sbin/munin11:30:242
155675499411,37cyclictest1620069-21/usr/sbin/munin10:10:162
155675499411,37cyclictest1573206-21/usr/sbin/munin07:55:162
155675499411,37cyclictest1565368-21/usr/sbin/munin07:30:262
155675499411,35cyclictest1638752-21apt-get11:05:152
155675499411,35cyclictest1571610-21memory07:50:202
155675499411,35cyclictest1557066-21/usr/sbin/munin07:10:172
155675499411,34cyclictest1662831-21apt-get12:15:122
155675499410,36cyclictest0-21swapper/212:35:122
155675499403,34cyclictest1562018-21users07:20:242
155675499402,35cyclictest1587267-21/usr/sbin/munin08:35:182
155675499402,32cyclictest1626809-21apt-get10:30:132
1556754994019,17cyclictest0-21swapper/210:55:152
155675499401,36cyclictest1623514-21df10:20:132
155675499401,36cyclictest1569907-21/usr/sbin/munin07:45:182
155675499401,36cyclictest1559210-21/usr/sbin/munin07:15:122
155675499401,34cyclictest1658574-21/usr/sbin/munin12:00:232
155675499401,34cyclictest1658574-21/usr/sbin/munin12:00:232
155675499401,34cyclictest1642259-21/usr/sbin/munin11:15:142
155675499401,33cyclictest1593753-21apt-get08:55:132
155675499401,33cyclictest1593753-21apt-get08:55:132
155675499400,34cyclictest0-21swapper/212:25:122
155675499400,33cyclictest0-21swapper/211:00:122
1556745994022,15cyclictest0-21swapper/011:25:010
1556759993929,6cyclictest1595216-21cron09:00:003
1556759993929,6cyclictest1595216-21cron08:59:593
155675499391,35cyclictest1654490-21/usr/sbin/munin11:50:172
155675499391,35cyclictest1647162-21cron11:29:012
155675499391,35cyclictest1604454-21/usr/sbin/munin09:25:182
155675499391,34cyclictest1640451-21apt-get11:10:142
155675499391,33cyclictest1566382-21/usr/sbin/munin07:35:192
155675499391,33cyclictest1564289-21latency_hist07:30:022
155675499391,32cyclictest1633628-21apt-get10:50:102
155675499390,34cyclictest0-21swapper/210:05:132
155675499390,34cyclictest0-21swapper/208:10:112
155675499390,33cyclictest0-21swapper/210:15:102
1556750993932,4cyclictest1650661-21munin-plugin-st11:40:001
1556750993910,14cyclictest24-21ksoftirqd/112:15:001
1556745993932,4cyclictest1609096-21munin-plugin-st09:40:000
1556759993829,6cyclictest1563708-21unixbench_singl07:25:273
1556759993828,6cyclictest1616143-21sendmail10:00:003
155675499382,33cyclictest1644944-21/usr/sbin/munin11:20:242
155675499381,34cyclictest1661830-21/usr/sbin/munin12:10:262
155675499381,34cyclictest1656585-21ntp_states11:55:232
155675499381,34cyclictest1616586-21apt-get10:00:152
155675499381,34cyclictest1606195-21/usr/sbin/munin09:30:162
155675499381,34cyclictest1597575-21/usr/sbin/munin09:05:202
155675499381,34cyclictest1590792-21/usr/sbin/munin08:45:202
155675499381,34cyclictest1581965-21/usr/sbin/munin08:20:172
155675499381,32cyclictest1650905-21munin-node11:40:122
155675499380,33cyclictest1609089-21apt-get09:40:052
155675499380,33cyclictest0-21swapper/209:50:132
155675499380,33cyclictest0-21swapper/209:50:132
1556750993825,4cyclictest131rcu_preempt07:45:141
1556745993832,4cyclictest1615774-21kworker/u8:1+events_unbound10:06:010
1556745993822,5cyclictest131rcu_preempt07:57:010
1556759993729,5cyclictest1567519-21cut07:40:013
1556759993727,5cyclictest0-21swapper/308:20:013
155675499371,33cyclictest1589058-21/usr/sbin/munin08:40:202
155675499370,31cyclictest0-21swapper/212:05:112
1556750993729,5cyclictest215-21dbus-daemon07:20:241
1556750993729,4cyclictest1609108-21apt-get09:40:001
1556750993712,10cyclictest1564637-21if_eth007:30:171
1556745993730,5cyclictest1640097-21kworker/u8:3+events_unbound11:26:180
1556745993718,4cyclictest131rcu_preempt08:40:250
1556745993716,4cyclictest131rcu_preempt09:17:020
1556759993626,6cyclictest0-21swapper/311:30:253
1556759993625,8cyclictest1601844-21grep09:15:263
1556750993626,6cyclictest0-21swapper/107:15:021
1556750993623,8cyclictest0-21swapper/108:45:181
1556750993614,7cyclictest1597993-21cat09:05:261
155674599366,21cyclictest11-21ksoftirqd/012:25:110
1556745993626,6cyclictest0-21swapper/010:40:000
1556745993626,6cyclictest0-21swapper/007:30:260
1556745993617,10cyclictest131rcu_preempt10:55:130
1556745993614,5cyclictest131rcu_preempt10:15:120
1556759993528,4cyclictest0-21swapper/312:10:183
1556759993526,5cyclictest0-21swapper/308:40:283
1556759993524,7cyclictest1663818-21grep12:15:243
1556750993526,6cyclictest1664310-21latency_hist12:20:001
1556750993525,6cyclictest1624394-21grep10:20:251
1556750993524,7cyclictest1627854-21grep10:30:261
1556750993511,7cyclictest1606351-21date09:30:181
155674599359,18cyclictest131rcu_preempt10:30:400
1556745993528,4cyclictest0-21swapper/007:20:010
1556745993524,4cyclictest131rcu_preempt08:11:240
1556745993521,9cyclictest0-21swapper/007:20:150
1556745993520,4cyclictest131rcu_preempt10:10:340
1556745993520,4cyclictest131rcu_preempt08:35:220
1556745993512,15cyclictest11-21ksoftirqd/009:40:210
1556759993426,4cyclictest0-21swapper/308:50:143
1556759993425,4cyclictest0-21swapper/308:15:013
1556759993422,7cyclictest0-21swapper/309:35:273
1556750993425,6cyclictest1622743-21unixbench_singl10:15:231
1556750993424,6cyclictest0-21swapper/108:55:311
1556750993424,6cyclictest0-21swapper/108:55:311
1556750993423,7cyclictest0-21swapper/108:50:221
155674599347,7cyclictest11-21ksoftirqd/010:50:000
1556745993424,6cyclictest0-21swapper/012:10:270
1556745993423,6cyclictest0-21swapper/011:35:260
1556745993420,5cyclictest131rcu_preempt07:10:230
1556745993419,9cyclictest131rcu_preempt11:00:260
1556745993416,5cyclictest131rcu_preempt07:10:010
1556745993416,4cyclictest131rcu_preempt09:46:230
1556745993416,10cyclictest131rcu_preempt09:00:170
1556745993415,9cyclictest131rcu_preempt12:21:540
1556745993415,4cyclictest131rcu_preempt11:40:080
1556745993414,10cyclictest131rcu_preempt08:55:130
1556745993414,10cyclictest131rcu_preempt08:55:120
1556759993325,4cyclictest1645554-21apt-key11:25:013
1556759993325,4cyclictest0-21swapper/309:40:273
1556759993323,7cyclictest1667721-21sh12:30:003
1556759993322,7cyclictest0-21swapper/307:55:013
1556759993321,7cyclictest1614433-21cut09:55:003
1556759993321,7cyclictest1614433-21cut09:54:593
1556759993321,7cyclictest0-21swapper/312:00:003
1556759993320,9cyclictest1627878-21unixbench_singl10:30:273
155675099339,5cyclictest24-21ksoftirqd/110:15:011
155675099338,5cyclictest24-21ksoftirqd/108:35:011
1556750993325,4cyclictest0-21swapper/108:10:011
1556750993324,6cyclictest0-21swapper/110:40:261
1556750993324,4cyclictest0-21swapper/108:45:011
1556750993322,7cyclictest0-21swapper/111:15:291
155674599333,19cyclictest11-21ksoftirqd/008:25:010
1556745993324,6cyclictest0-21swapper/010:00:010
1556745993320,5cyclictest131rcu_preempt07:50:180
1556745993317,10cyclictest131rcu_preempt07:40:170
1556745993315,4cyclictest131rcu_preempt09:24:430
1556745993315,10cyclictest131rcu_preempt09:25:250
1556745993313,11cyclictest131rcu_preempt11:30:150
1556745993312,11cyclictest0-21swapper/010:44:390
1556745993312,11cyclictest0-21swapper/008:29:500
1556745993310,11cyclictest131rcu_preempt11:15:170
1556759993225,5cyclictest1569047-21kworker/u8:1+events_unbound07:45:293
1556759993224,4cyclictest0-21swapper/310:35:173
1556759993223,5cyclictest0-21swapper/309:45:583
1556759993221,7cyclictest1631279-21grep10:40:263
155675099328,5cyclictest24-21ksoftirqd/110:00:251
155675099328,5cyclictest24-21ksoftirqd/107:50:201
1556750993224,4cyclictest0-21swapper/112:40:001
155674599329,16cyclictest11-21ksoftirqd/008:10:020
155674599328,17cyclictest131rcu_preempt07:25:190
1556745993226,4cyclictest1574168-21kworker/u8:3+events_unbound08:17:420
1556745993221,5cyclictest131rcu_preempt08:00:020
1556745993221,4cyclictest131rcu_preempt11:50:190
1556745993219,4cyclictest131rcu_preempt12:32:470
1556745993218,4cyclictest131rcu_preempt12:15:190
1556745993216,5cyclictest131rcu_preempt10:25:240
1556745993214,5cyclictest131rcu_preempt09:51:060
1556745993214,5cyclictest131rcu_preempt09:51:050
1556745993214,10cyclictest131rcu_preempt12:05:220
1556745993213,4cyclictest0-21swapper/012:37:540
1556745993211,10cyclictest131rcu_preempt08:30:150
1556745993210,5cyclictest131rcu_preempt09:10:150
1556745993210,11cyclictest0-21swapper/009:31:120
1556759993123,4cyclictest0-21swapper/311:05:273
1556759993121,5cyclictest0-21swapper/312:30:153
155675099319,6cyclictest24-21ksoftirqd/109:25:151
155675099319,4cyclictest24-21ksoftirqd/107:55:181
1556750993124,3cyclictest0-21swapper/107:45:011
1556750993122,5cyclictest0-21swapper/112:35:001
1556750993118,9cyclictest215-21dbus-daemon08:10:391
155674599319,14cyclictest11-21ksoftirqd/011:10:210
1556745993120,4cyclictest131rcu_preempt12:00:120
1556745993120,4cyclictest131rcu_preempt12:00:120
1556745993117,4cyclictest131rcu_preempt09:05:160
1556745993113,11cyclictest131rcu_preempt11:45:150
1556745993112,11cyclictest131rcu_preempt08:45:210
1556745993111,5cyclictest0-21swapper/007:48:360
1556745993110,14cyclictest11-21ksoftirqd/010:20:180
1556759993022,4cyclictest0-21swapper/309:00:153
1556759993020,8cyclictest391ktimers/308:45:153
1556759993020,7cyclictest0-21swapper/311:20:013
1556759993018,8cyclictest0-21swapper/310:00:273
155675099302,9cyclictest1653061-21sshd11:45:211
1556750993024,3cyclictest0-21swapper/110:50:131
1556750993022,4cyclictest0-21swapper/109:25:001
1556750993020,6cyclictest0-21swapper/110:55:251
1556750993020,6cyclictest0-21swapper/109:45:271
1556750993020,6cyclictest0-21swapper/109:10:211
1556745993015,4cyclictest131rcu_preempt11:05:150
1556745993012,4cyclictest131rcu_preempt07:37:350
155675099297,4cyclictest24-21ksoftirqd/109:43:231
155675099297,4cyclictest24-21ksoftirqd/108:25:241
155675099294,7cyclictest24-21ksoftirqd/109:55:011
155675099294,7cyclictest24-21ksoftirqd/109:55:011
1556750992919,7cyclictest1601788-21grep09:15:251
1556750992919,6cyclictest0-21swapper/110:45:271
1556745992914,4cyclictest131rcu_preempt10:50:220
1556745992914,4cyclictest131rcu_preempt10:00:420
1556759992819,5cyclictest0-21swapper/312:06:203
1556759992819,4cyclictest0-21swapper/311:45:173
1556759992818,6cyclictest0-21swapper/309:10:273
1556759992817,7cyclictest0-21swapper/307:15:263
1556759992816,8cyclictest1606938-21grep09:30:253
155675099285,5cyclictest24-21ksoftirqd/111:00:231
1556750992820,4cyclictest0-21swapper/112:05:161
1556750992820,4cyclictest0-21swapper/111:20:251
1556750992820,4cyclictest0-21swapper/107:25:171
1556750992817,7cyclictest0-21swapper/112:25:251
1556759992720,4cyclictest0-21swapper/311:10:263
1556759992719,4cyclictest0-21swapper/311:25:263
1556759992719,4cyclictest0-21swapper/310:55:243
1556759992719,4cyclictest0-21swapper/309:20:413
155675099279,10cyclictest131rcu_preempt10:09:081
155675099275,5cyclictest24-21ksoftirqd/111:40:141
155675099275,4cyclictest24-21ksoftirqd/111:25:021
1556750992720,4cyclictest231ktimers/111:11:421
1556750992717,6cyclictest0-21swapper/109:00:281
1556750992713,5cyclictest131rcu_preempt11:50:441
1556759992619,4cyclictest0-21swapper/310:10:283
1556759992618,4cyclictest0-21swapper/308:25:173
155675099265,6cyclictest0-21swapper/112:00:101
155675099265,6cyclictest0-21swapper/112:00:101
155675099265,5cyclictest24-21ksoftirqd/111:55:221
1556750992620,3cyclictest0-21swapper/108:15:261
1556750992618,4cyclictest0-21swapper/108:00:251
1556750992615,6cyclictest0-21swapper/110:35:261
1556750992612,10cyclictest0-21swapper/109:55:221
1556750992610,3cyclictest131rcu_preempt11:08:451
1556759992517,4cyclictest0-21swapper/310:45:163
1556750992517,5cyclictest0-21swapper/111:30:251
1556759992417,5cyclictest1572559-21kworker/u8:1+events_unbound07:55:333
1556759992415,6cyclictest391ktimers/308:00:083
1556759992415,5cyclictest0-21swapper/310:55:003
1556759992415,5cyclictest0-21swapper/308:30:233
155675099243,7cyclictest24-21ksoftirqd/112:20:211
155675099242,14cyclictest24-21ksoftirqd/107:39:031
1556750992410,9cyclictest131rcu_preempt10:25:101
1556759992316,4cyclictest1665092-21ntpq12:20:203
1556759992316,3cyclictest0-21swapper/309:10:003
1556759992311,7cyclictest0-21swapper/308:20:273
155675999230,3cyclictest0-21swapper/307:23:113
1556759992214,4cyclictest0-21swapper/310:25:263
1556759992213,6cyclictest391ktimers/310:20:253
1556759992115,3cyclictest1604907-21sendmail_mailqu09:25:243
1556759992114,5cyclictest391ktimers/310:05:213
1556759992113,6cyclictest391ktimers/311:35:123
1556759992113,4cyclictest0-21swapper/307:30:213
1556759992111,8cyclictest1651307-21sed11:40:193
1556759992014,3cyclictest0-21swapper/312:00:213
1556759992014,3cyclictest0-21swapper/312:00:203
1556759992014,3cyclictest0-21swapper/308:35:203
1556759992012,5cyclictest1568199-21/usr/sbin/munin07:40:183
1556759992012,4cyclictest1655778-21sh11:55:003
1556759992012,4cyclictest0-21swapper/311:00:263
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional