You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-09 - 01:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot5s.osadl.org (updated Mon Jun 08, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1578254995543,7cyclictest1630340-21grep09:35:262
1578243995144,4cyclictest0-21swapper/010:10:120
1578250995044,4cyclictest1637254-21kworker/u8:0+events_unbound10:00:521
1578243994937,7cyclictest1621846-21grep09:10:250
1578257994740,5cyclictest1622209-21kworker/u8:2+events_unbound09:15:453
1578257994740,5cyclictest1622209-21kworker/u8:2+events_unbound09:15:453
1578254994729,13cyclictest0-21swapper/212:19:592
1578243994739,5cyclictest1625244-21unixbench_multi09:20:250
1578243994739,5cyclictest1625244-21unixbench_multi09:20:240
1578243994733,10cyclictest121ktimers/007:55:130
1578257994638,6cyclictest1622209-21kworker/u8:2+events_unbound09:30:483
1578254994535,7cyclictest1679212-21munin-plugin-st12:00:002
1578257994433,5cyclictest1652902-21kworker/u8:1+events_unbound10:45:593
157825499441,9cyclictest1601094-21/usr/sbin/munin08:10:252
157825499441,9cyclictest1601094-21/usr/sbin/munin08:10:252
1578257994333,7cyclictest1590692-21grep07:40:263
157825499439,30cyclictest1689688-21ls12:30:012
157825499431,7cyclictest1638650-21/usr/sbin/munin10:00:242
157825499431,6cyclictest1681274-21df_inode12:05:162
157825499431,11cyclictest1586255-21apt-get07:30:132
157825499431,10cyclictest1624226-21/usr/sbin/munin09:20:262
157825499431,10cyclictest1624226-21/usr/sbin/munin09:20:262
1578250994335,4cyclictest1685626-21unixbench_singl12:15:261
1578257994236,4cyclictest1632463-21kworker/u8:1+events_unbound09:50:493
1578257994235,5cyclictest1625506-21kworker/u8:1+events_unbound09:25:483
1578257994231,8cyclictest1693079-21sendmail12:39:593
1578254994234,4cyclictest1671978-21sendmail11:40:002
1578254994232,7cyclictest311ktimers/212:35:132
157825499421,9cyclictest1585386-21/usr/sbin/munin07:25:262
157825499411,9cyclictest1687785-21cron12:23:002
157825499411,9cyclictest1674967-21/usr/sbin/munin11:45:262
157825499411,8cyclictest1646778-21/usr/sbin/munin10:25:182
157825499411,8cyclictest1592159-21/usr/sbin/munin07:45:232
157825499411,4cyclictest1645392-21find10:20:222
157825499411,35cyclictest1633017-21cat09:45:162
157825499410,3cyclictest0-21swapper/208:35:122
1578257994022,5cyclictest131rcu_preempt12:30:003
157825499404,31cyclictest1655675-21tr10:50:202
1578254994029,7cyclictest1596424-21apt-key08:00:012
157825499402,5cyclictest1660262-21apt-get11:05:122
157825499402,33cyclictest1683019-21fschecks_count12:10:152
157825499402,11cyclictest1598356-21sed08:05:012
157825499401,9cyclictest1676151-21/usr/sbin/munin11:50:142
157825499401,9cyclictest1623585-21users09:15:262
157825499401,9cyclictest1623585-21users09:15:262
157825499401,8cyclictest1654453-21unixbench_multi10:45:262
157825499401,7cyclictest1611091-21/usr/sbin/munin08:40:232
157825499401,3cyclictest0-21swapper/209:30:112
157825499401,12cyclictest1658342-21awk11:00:012
157825499400,3cyclictest0-21swapper/207:15:112
157825499400,35cyclictest1588072-21df_inode07:35:152
1578257993933,4cyclictest1685684-21kworker/u8:1+events_unbound12:21:123
1578257993933,4cyclictest1663060-21kworker/u8:2+events_unbound11:41:073
157825499392,30cyclictest1612253-21cut08:45:152
157825499391,9cyclictest1582546-21/usr/sbin/munin07:20:142
157825499391,8cyclictest1620013-21/usr/sbin/munin09:05:252
157825499391,7cyclictest1616130-21meminfo08:55:192
157825499391,4cyclictest1639578-21apt-get10:05:122
157825499391,3cyclictest0-21swapper/209:25:132
1578250993933,4cyclictest1661348-21kworker/u8:0+events_unbound11:06:031
1578250993917,12cyclictest1623033-21ntpq09:15:201
1578250993917,12cyclictest1623033-21ntpq09:15:191
1578257993817,13cyclictest131rcu_preempt08:15:233
1578257993817,13cyclictest131rcu_preempt08:15:233
1578257993816,14cyclictest131rcu_preempt10:55:173
1578257993816,13cyclictest131rcu_preempt08:00:023
1578257993815,6cyclictest131rcu_preempt12:00:113
157825499383,7cyclictest1607339-21sed08:30:182
157825499382,4cyclictest1658680-21df_inode11:00:172
157825499382,32cyclictest1642739-21cron10:15:002
157825499381,9cyclictest1598986-21/usr/sbin/munin08:05:222
157825499381,7cyclictest1690071-21grep12:30:152
157825499381,6cyclictest1672265-21/usr/sbin/munin11:40:142
157825499381,6cyclictest1613536-21apt-get08:50:062
157825499381,3cyclictest0-21swapper/211:10:112
157825499381,3cyclictest0-21swapper/209:00:132
157825499381,3cyclictest0-21swapper/208:25:122
157825499381,34cyclictest1635021-21ntp_states09:50:192
157825499381,15cyclictest1593088-21chrt07:50:122
157825499381,11cyclictest1648795-21fgrep10:30:232
157825499381,11cyclictest1621869-21awk09:10:252
1578250993829,6cyclictest0-21swapper/112:15:001
1578257993731,4cyclictest1615359-21kworker/u8:2+events_unbound08:57:413
157825499371,8cyclictest1663827-21fschecks_count11:15:132
157825499371,7cyclictest1652062-21if_eth110:40:172
157825499371,5cyclictest1602268-21meminfo08:15:192
157825499371,5cyclictest1602268-21meminfo08:15:182
157825499371,4cyclictest1665443-21/usr/sbin/munin11:20:152
157825499371,3cyclictest1631051-21apt-get09:40:112
157825499371,3cyclictest1604455-21/usr/sbin/munin08:20:262
157825499371,3cyclictest0-21swapper/211:30:122
157825499371,3cyclictest0-21swapper/210:35:162
157825499371,3cyclictest0-21swapper/210:15:122
157825499371,3cyclictest0-21swapper/209:55:142
157825499371,32cyclictest1579458-21unixbench_singl07:10:272
1578250993718,10cyclictest131rcu_preempt08:30:011
1578243993730,4cyclictest1691351-21cron12:34:590
1578243993730,3cyclictest1627403-21kthreadd09:30:000
1578257993630,4cyclictest1680891-21kworker/u8:2+events_unbound12:11:093
1578257993619,5cyclictest131rcu_preempt09:40:173
1578257993619,5cyclictest131rcu_preempt08:25:193
1578257993611,4cyclictest131rcu_preempt07:15:043
157825499361,10cyclictest1667965-21wc11:25:232
157825499360,3cyclictest0-21swapper/207:40:122
1578250993629,4cyclictest1641051-21idleruntime-cro10:10:011
1578243993628,4cyclictest0-21swapper/007:40:010
1578257993519,5cyclictest131rcu_preempt11:50:013
1578257993517,5cyclictest131rcu_preempt10:30:243
1578257993516,12cyclictest131rcu_preempt10:25:223
1578257993514,4cyclictest131rcu_preempt08:45:133
157825499353,7cyclictest1679891-21latency12:00:182
1578257993427,5cyclictest1663288-21kworker/u8:3+events_unbound11:21:053
1578257993426,6cyclictest1592735-21kworker/u8:0+events_unbound07:50:223
1578257993426,4cyclictest0-21swapper/310:35:303
1578257993418,5cyclictest131rcu_preempt08:10:163
1578257993418,5cyclictest131rcu_preempt08:10:163
1578257993417,5cyclictest131rcu_preempt10:20:243
1578257993416,9cyclictest131rcu_preempt12:05:133
1578257993416,5cyclictest131rcu_preempt07:45:213
1578257993415,5cyclictest131rcu_preempt11:15:233
1578257993415,10cyclictest131rcu_preempt09:35:273
1578257993413,12cyclictest131rcu_preempt07:10:173
157825099348,6cyclictest131rcu_preempt10:35:161
1578250993426,5cyclictest231ktimers/108:20:111
1578250993426,4cyclictest0-21swapper/108:10:251
1578250993426,4cyclictest0-21swapper/108:10:251
1578243993425,5cyclictest0-21swapper/010:15:250
1578257993321,5cyclictest131rcu_preempt08:30:113
1578257993318,4cyclictest131rcu_preempt07:30:143
1578257993317,5cyclictest131rcu_preempt09:10:163
1578257993317,5cyclictest131rcu_preempt08:35:183
1578257993316,5cyclictest131rcu_preempt11:30:193
1578257993315,5cyclictest131rcu_preempt08:40:243
1578257993314,5cyclictest131rcu_preempt09:55:263
1578257993314,12cyclictest131rcu_preempt08:50:253
1578257993312,4cyclictest131rcu_preempt11:55:083
1578257993312,13cyclictest131rcu_preempt10:15:183
1578250993325,4cyclictest0-21swapper/110:55:151
1578250993324,5cyclictest0-21swapper/112:10:001
1578250993323,7cyclictest1630823-21sendmail09:40:001
1578250993323,6cyclictest0-21swapper/111:20:001
1578250993322,7cyclictest1583441-21grep07:20:261
1578250993320,6cyclictest1663288-21kworker/u8:3+events_unbound11:21:441
1578250993315,6cyclictest131rcu_preempt07:15:111
157825099331,4cyclictest0-21swapper/112:30:121
1578250993313,6cyclictest131rcu_preempt08:15:121
1578250993313,6cyclictest131rcu_preempt08:15:121
1578250993310,3cyclictest131rcu_preempt09:50:111
1578243993326,4cyclictest0-21swapper/010:30:250
1578257993225,5cyclictest1637254-21kworker/u8:0+events_unbound10:10:533
1578257993214,5cyclictest131rcu_preempt11:00:183
1578257993213,5cyclictest131rcu_preempt08:00:193
1578257993212,11cyclictest131rcu_preempt11:05:233
1578257993212,11cyclictest131rcu_preempt10:55:013
1578257993211,13cyclictest131rcu_preempt12:15:253
1578257993210,11cyclictest131rcu_preempt09:00:153
1578250993226,4cyclictest1609896-21kworker/u8:0+events_unbound08:50:411
1578250993222,6cyclictest1633814-21grep09:45:251
1578243993226,4cyclictest1679209-21munin-run12:00:000
1578243993225,4cyclictest0-21swapper/007:50:010
1578257993119,9cyclictest391ktimers/307:35:123
1578257993116,5cyclictest1668415-21kworker/u8:1+events_unbound11:39:193
1578257993115,7cyclictest1630710-21kworker/u8:0+events_unbound09:47:093
1578257993115,5cyclictest131rcu_preempt12:30:153
1578257993112,4cyclictest131rcu_preempt10:05:303
1578257993111,12cyclictest40-21ksoftirqd/307:20:213
1578257993110,13cyclictest40-21ksoftirqd/310:00:173
1578257993110,12cyclictest131rcu_preempt10:40:253
1578257993110,11cyclictest40-21ksoftirqd/308:20:233
1578250993124,4cyclictest1641689-21latency10:10:181
1578250993123,4cyclictest0-21swapper/109:55:261
1578250993123,4cyclictest0-21swapper/109:40:141
1578250993123,4cyclictest0-21swapper/109:20:151
1578250993123,4cyclictest0-21swapper/109:20:151
1578250993121,6cyclictest0-21swapper/107:40:171
1578250993111,7cyclictest131rcu_preempt10:45:121
1578250993111,4cyclictest19950systemd-network07:25:061
1578243993121,6cyclictest0-21swapper/007:25:020
157825799307,5cyclictest40-21ksoftirqd/311:45:013
157825799307,14cyclictest40-21ksoftirqd/311:25:223
157825799302,10cyclictest40-21ksoftirqd/311:10:103
1578257993017,6cyclictest1622209-21kworker/u8:2+events_unbound09:24:203
1578257993017,6cyclictest1622209-21kworker/u8:2+events_unbound09:24:203
1578257993012,5cyclictest131rcu_preempt09:05:253
1578257993010,5cyclictest131rcu_preempt07:30:013
1578250993025,3cyclictest645-21runrttasks09:30:101
1578250993024,4cyclictest1689496-21kworker/u8:1+events_unbound12:36:141
1578250993022,4cyclictest0-21swapper/108:35:261
1578250993022,4cyclictest0-21swapper/108:30:151
1578250993020,4cyclictest131rcu_preempt08:48:141
1578250993010,6cyclictest131rcu_preempt09:25:111
1578243993023,3cyclictest1658311-21kthreadd11:00:000
1578243993023,3cyclictest0-21swapper/008:40:010
1578243993022,5cyclictest1684316-21cron12:15:000
1578257992915,3cyclictest131rcu_preempt08:05:263
157825099298,12cyclictest131rcu_preempt09:00:181
1578250992920,6cyclictest0-21swapper/112:30:001
1578250992918,6cyclictest0-21swapper/107:50:151
1578250992917,6cyclictest1668415-21kworker/u8:1+events_unbound11:45:531
1578250992915,4cyclictest131rcu_preempt08:40:131
157824399292,4cyclictest131rcu_preempt07:15:130
1578243992923,4cyclictest1647732-21kworker/u8:2+events_unbound10:35:560
1578243992923,3cyclictest0-21swapper/011:35:260
1578243992920,6cyclictest0-21swapper/010:50:250
157824399290,27cyclictest0-21swapper/011:50:110
157825099284,14cyclictest24-21ksoftirqd/108:09:471
1578250992822,4cyclictest1668415-21kworker/u8:1+events_unbound11:40:201
1578250992817,4cyclictest131rcu_preempt12:00:161
1578250992815,4cyclictest131rcu_preempt07:30:121
157825099281,17cyclictest1659415-21grep11:00:271
1578243992817,7cyclictest1685556-21grep12:15:260
1578243992812,13cyclictest0-21swapper/011:05:000
157825099275,12cyclictest24-21ksoftirqd/111:30:251
1578250992720,4cyclictest0-21swapper/107:45:251
1578250992718,7cyclictest1663060-21kworker/u8:2+events_unbound11:28:451
1578250992715,4cyclictest131rcu_preempt10:20:211
1578250992714,7cyclictest131rcu_preempt11:50:011
1578250992713,6cyclictest1590807-21kworker/u8:1+events_unbound08:04:201
1578250992710,11cyclictest131rcu_preempt09:05:221
1578250992710,10cyclictest131rcu_preempt11:15:011
1578243992721,3cyclictest0-21swapper/007:30:010
1578243992719,6cyclictest0-21swapper/009:55:260
1578250992620,4cyclictest1654680-21kworker/u8:0+events_unbound10:51:011
1578250992620,4cyclictest1637254-21kworker/u8:0+events_unbound10:30:411
1578250992620,4cyclictest1621435-21proc_pri09:10:231
1578250992620,4cyclictest1594527-21kworker/u8:2+events_unbound07:56:581
1578250992615,4cyclictest131rcu_preempt07:35:251
1578243992617,5cyclictest1630811-21cat09:40:000
1578243992614,9cyclictest1675702-21sed11:50:010
1578250992519,4cyclictest1576163-21kworker/u8:0+events_unbound07:10:181
1578250992519,3cyclictest0-21swapper/112:20:151
1578250992517,6cyclictest231ktimers/108:55:131
1578250992517,5cyclictest0-21swapper/111:35:261
1578250992517,5cyclictest0-21swapper/110:25:141
1578243992518,4cyclictest1654913-21munin-plugin-st10:50:010
1578243992517,5cyclictest1665134-21latency_hist11:20:000
157825099243,4cyclictest24-21ksoftirqd/111:55:161
1578250992417,5cyclictest1637254-21kworker/u8:0+events_unbound10:15:191
1578250992416,3cyclictest131rcu_preempt10:40:211
1578243992417,5cyclictest1615359-21kworker/u8:2+events_unbound09:06:410
1578243992417,4cyclictest0-21swapper/010:25:240
1578243992417,3cyclictest0-21swapper/009:00:420
1578243992416,5cyclictest1-21systemd09:15:440
1578243992416,5cyclictest1-21systemd09:15:440
1578243992415,6cyclictest1667640-21ntpq11:25:190
1578243992413,7cyclictest0-21swapper/009:30:230
157824399241,19cyclictest0-21swapper/012:38:080
157824399241,19cyclictest0-21swapper/011:24:280
157824399241,19cyclictest0-21swapper/009:42:360
1578243992319,2cyclictest13550irq/42-30be0000.ethernet08:10:220
1578243992319,2cyclictest13550irq/42-30be0000.ethernet08:10:210
1578243992318,2cyclictest0-21swapper/009:55:010
1578243992317,3cyclictest0-21swapper/008:15:150
1578243992317,3cyclictest0-21swapper/008:15:150
1578243992317,3cyclictest0-21swapper/007:30:240
1578243992316,5cyclictest1602988-21kworker/u8:2+events_unbound08:25:330
1578243992316,3cyclictest0-21swapper/011:07:220
1578243992316,3cyclictest0-21swapper/010:23:590
1578243992316,3cyclictest0-21swapper/007:44:510
1578243992315,4cyclictest1612278-21fschecks_time08:45:150
1578243992314,5cyclictest121ktimers/008:10:010
1578243992312,8cyclictest1669681-21cat11:30:240
157824399231,18cyclictest0-21swapper/012:01:150
157824399230,19cyclictest0-21swapper/011:12:030
157824399230,19cyclictest0-21swapper/009:46:330
157824399230,19cyclictest0-21swapper/008:59:240
157824399230,19cyclictest0-21swapper/008:21:430
157824399230,19cyclictest0-21swapper/007:50:060
157824399230,19cyclictest0-21swapper/007:13:120
1578243992216,4cyclictest0-21swapper/008:00:100
1578243992216,3cyclictest1651832-21ls10:40:130
1578243992215,3cyclictest0-21swapper/012:21:180
1578243992215,3cyclictest0-21swapper/010:05:240
1578243992116,3cyclictest1638082-21fschecks_time10:00:150
1578243992116,3cyclictest13550irq/42-30be0000.ethernet12:25:110
1578243992114,5cyclictest121ktimers/011:40:150
1578243992114,5cyclictest121ktimers/008:50:260
1578243992114,4cyclictest121ktimers/012:05:270
157824399211,3cyclictest0-21swapper/008:40:080
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional