You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-21 - 19:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Fri Aug 21, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
155689699536,18cyclictest1579405-21unixbench_singl08:10:252
1556896995327,21cyclictest1598519-21grep09:05:262
1556901994838,7cyclictest1632472-21fschecks_time10:45:153
1556893994835,8cyclictest1560847-21sendmail07:20:011
1556893994731,12cyclictest231ktimers/108:15:131
1556893994435,5cyclictest1656143-21munin-plugin-st11:55:001
1556893994435,5cyclictest1656143-21munin-plugin-st11:55:001
1556901994333,6cyclictest1658298-21fschecks_time12:00:163
155689699423,8cyclictest1591647-21/usr/sbin/munin08:45:242
155689699421,9cyclictest1646762-21/usr/sbin/munin11:25:222
155689699421,10cyclictest1608998-21/usr/sbin/munin09:35:252
155689699421,10cyclictest1590190-21sshd08:41:432
1556896994210,28cyclictest1639075-21apt-key11:05:012
1556888994235,4cyclictest1583448-21idleruntime-cro08:25:010
1556896994132,6cyclictest0-21swapper/212:20:002
155689699412,8cyclictest1584801-21cron08:26:012
155689699411,9cyclictest1628771-21diskstats10:35:152
155689699411,11cyclictest1570423-21/usr/sbin/munin07:45:232
155689699403,8cyclictest1622210-21/usr/sbin/munin10:15:212
155689699401,9cyclictest1559878-21/usr/sbin/munin07:15:202
155689699401,7cyclictest1641351-21interrupts11:10:172
1556896994016,21cyclictest0-21swapper/210:10:162
1556896994011,25cyclictest1611289-21latency_hist09:45:002
1556893994031,5cyclictest0-21swapper/110:30:151
1556893994028,9cyclictest231ktimers/112:15:111
1556888994033,4cyclictest1623096-21cron10:20:000
1556901993929,7cyclictest1614146-21unixbench_singl09:50:253
1556896993928,7cyclictest1619250-21grep10:05:272
155689699391,8cyclictest1664905-21/usr/sbin/munin12:20:102
155689699391,8cyclictest1662113-21/usr/sbin/munin12:10:212
155689699391,8cyclictest1639590-21cat11:05:162
155689699391,6cyclictest1634891-21/usr/sbin/munin10:50:232
155689699391,5cyclictest1658616-21/usr/sbin/munin12:00:212
155689699391,5cyclictest1630823-21/usr/sbin/munin10:40:192
1556896993915,20cyclictest0-21swapper/210:30:252
155689699391,11cyclictest1614915-21diskstats09:55:132
1556896993910,24cyclictest1648903-21unixbench_singl11:30:242
1556893993925,10cyclictest231ktimers/107:50:111
155689699388,27cyclictest1645551-21seq11:20:492
155689699382,5cyclictest1625612-21ntp_states10:25:202
155689699382,4cyclictest1596281-21memory09:00:202
155689699381,9cyclictest1660186-21/usr/sbin/munin12:05:192
155689699381,9cyclictest1571618-21seq07:50:132
155689699381,6cyclictest1605621-21grep09:25:262
155689699381,5cyclictest1636134-21memory10:55:192
155689699381,5cyclictest1557162-21df_inode07:10:142
155689699381,4cyclictest1586109-21ntp_kernel_err08:30:242
155689699381,4cyclictest1575694-21/usr/sbin/munin08:00:242
155689699381,4cyclictest1568685-21ntp_states07:40:202
155689699381,4cyclictest1564977-21/usr/sbin/munin07:30:162
155689699381,4cyclictest1564977-21/usr/sbin/munin07:30:152
155689699381,3cyclictest1670585-21/usr/sbin/munin12:35:192
155689699381,3cyclictest1574195-21/usr/sbin/munin07:55:272
155689699381,10cyclictest1567089-21grep07:35:252
155689699381,10cyclictest1567089-21grep07:35:242
1556893993829,5cyclictest0-21swapper/111:40:151
1556893993826,8cyclictest231ktimers/111:55:141
1556888993831,3cyclictest0-21swapper/011:15:240
1556888993825,9cyclictest1671008-21grep12:35:230
1556901993727,6cyclictest0-21swapper/307:25:003
155689699379,23cyclictest0-21swapper/209:50:242
155689699373,9cyclictest53-21kcompactd010:45:282
155689699372,6cyclictest1656476-21/usr/sbin/munin11:55:172
155689699371,9cyclictest1649647-21/usr/sbin/munin11:35:132
155689699371,9cyclictest1642836-21diskstats11:15:132
155689699371,9cyclictest1624418-21users10:20:252
155689699371,9cyclictest1561194-21/usr/sbin/munin07:20:132
155689699371,8cyclictest1653486-21/usr/sbin/munin11:45:212
155689699371,8cyclictest1651412-21cut11:40:152
155689699371,7cyclictest1592736-21/usr/sbin/munin08:50:182
155689699371,7cyclictest1588356-21head08:35:262
155689699371,5cyclictest1606847-21ntp_states09:30:222
155689699371,5cyclictest1606847-21ntp_states09:30:222
155689699371,4cyclictest1603953-21users09:20:272
155689699371,3cyclictest1601504-21ntp_states09:15:212
155689699371,3cyclictest1562983-21apt-get07:25:062
1556888993730,4cyclictest1640321-21unixbench_multi11:05:230
1556888993729,4cyclictest0-21swapper/007:28:400
1556901993626,6cyclictest0-21swapper/308:35:003
155689699363,8cyclictest1594617-21sed08:55:222
155689699361,9cyclictest1668303-21munin-node12:30:102
155689699361,9cyclictest1617144-21grep10:00:232
155689699361,9cyclictest1580779-21tr08:15:232
155689699361,7cyclictest1577087-21meminfo08:05:192
155689699361,4cyclictest1666602-21/usr/sbin/munin12:25:242
155689699361,4cyclictest1611968-21ntp_states09:45:212
1556888993630,3cyclictest13550irq/42-30be0000.ethernet11:50:270
1556888993630,3cyclictest13550irq/42-30be0000.ethernet11:50:270
1556888993628,4cyclictest0-21swapper/009:40:000
1556901993524,6cyclictest0-21swapper/309:35:243
1556901993523,7cyclictest0-21swapper/309:45:253
155689699351,5cyclictest1582686-21grep08:20:212
155689699351,10cyclictest1599700-21cat09:10:182
1556893993529,4cyclictest1583250-21kworker/u8:3+events_unbound08:43:511
1556893993527,4cyclictest0-21swapper/109:50:151
1556888993527,4cyclictest0-21swapper/008:30:000
1556888993525,6cyclictest0-21swapper/012:25:010
155689699343,4cyclictest1655113-21/usr/sbin/munin11:50:202
155689699343,4cyclictest1655113-21/usr/sbin/munin11:50:202
1556888993426,4cyclictest0-21swapper/009:10:230
1556888993424,6cyclictest0-21swapper/009:05:250
1556901993325,4cyclictest0-21swapper/311:15:233
1556901993324,6cyclictest1657831-21cron12:00:003
1556901993323,6cyclictest0-21swapper/307:45:273
1556901993322,7cyclictest0-21swapper/307:40:243
1556901993322,6cyclictest0-21swapper/307:30:013
1556888993326,5cyclictest1567845-21kworker/u8:0+events_unbound07:58:420
1556888993322,7cyclictest0-21swapper/007:40:000
1556888993322,7cyclictest0-21swapper/007:40:000
1556901993224,4cyclictest0-21swapper/308:43:493
1556901993223,4cyclictest0-21swapper/311:35:243
1556901993222,5cyclictest0-21swapper/312:24:313
1556901993221,6cyclictest0-21swapper/310:05:263
1556901993220,7cyclictest0-21swapper/308:20:243
1556893993226,4cyclictest1612583-21kworker/u8:1+events_unbound09:49:041
155689399322,5cyclictest131rcu_preempt10:20:111
1556888993224,4cyclictest0-21swapper/012:25:230
1556888993224,4cyclictest0-21swapper/011:00:000
1556888993223,5cyclictest0-21swapper/011:45:260
155688899321,29cyclictest0-21swapper/011:35:110
1556901993123,4cyclictest0-21swapper/310:25:163
1556901993123,4cyclictest0-21swapper/308:05:243
1556893993124,5cyclictest1648826-21unixbench_multi11:30:231
1556893993123,4cyclictest0-21swapper/112:10:151
1556893993123,4cyclictest0-21swapper/111:00:131
1556893993121,7cyclictest1642482-21sh11:15:011
1556888993121,7cyclictest1658287-21grep12:00:160
1556888993121,6cyclictest0-21swapper/010:05:000
155688899312,11cyclictest1630114-21cron10:39:590
1556901993021,6cyclictest0-21swapper/308:25:233
1556901993020,6cyclictest1664214-21grep12:15:253
1556901993020,6cyclictest0-21swapper/309:30:263
1556901993020,6cyclictest0-21swapper/309:30:253
1556901993019,7cyclictest0-21swapper/312:35:233
1556901993018,7cyclictest0-21swapper/308:55:263
155689399302,3cyclictest131rcu_preempt08:35:141
1556893993023,5cyclictest1636858-21kworker/u8:1+events_unbound11:24:171
1556893993022,5cyclictest1571292-21munin-plugin-st07:50:001
1556893993022,4cyclictest0-21swapper/110:00:151
1556893993020,6cyclictest1595172-21grep08:55:261
1556888993023,3cyclictest0-21swapper/008:40:000
1556888993021,7cyclictest121ktimers/012:10:120
1556888993020,6cyclictest0-21swapper/010:20:260
1556901992922,3cyclictest0-21swapper/310:40:143
1556901992921,5cyclictest0-21swapper/308:35:163
1556901992921,4cyclictest0-21swapper/309:40:273
1556901992920,5cyclictest0-21swapper/310:15:003
1556901992919,6cyclictest1640319-21grep11:05:233
1556901992918,7cyclictest0-21swapper/311:45:003
1556901992918,7cyclictest0-21swapper/310:35:243
1556901992918,7cyclictest0-21swapper/307:30:163
1556901992918,7cyclictest0-21swapper/307:30:153
1556893992922,5cyclictest1631592-21kworker/u8:2+events_unbound10:44:121
1556893992922,5cyclictest1584974-21kworker/u8:0+events_unbound09:02:421
1556893992922,4cyclictest0-21swapper/109:35:251
1556893992919,6cyclictest0-21swapper/107:55:261
1556893992918,7cyclictest0-21swapper/110:59:131
1556893992917,8cyclictest1610736-21grep09:40:281
1556888992923,4cyclictest1579853-21latency_hist08:15:010
1556888992919,6cyclictest53-21kcompactd011:55:220
1556901992821,4cyclictest0-21swapper/309:05:003
1556901992820,4cyclictest0-21swapper/310:55:263
1556893992820,4cyclictest0-21swapper/108:25:141
1556893992818,7cyclictest1607284-21grep09:30:251
1556893992818,7cyclictest1607284-21grep09:30:251
155689399281,4cyclictest131rcu_preempt12:25:101
1556888992821,5cyclictest1584974-21kworker/u8:0+events_unbound10:14:080
1556888992820,4cyclictest0-21swapper/012:30:230
1556901992721,4cyclictest391ktimers/311:52:593
1556901992721,4cyclictest391ktimers/311:52:593
1556901992719,5cyclictest0-21swapper/311:20:273
1556901992719,4cyclictest0-21swapper/309:15:253
1556893992720,4cyclictest0-21swapper/112:00:251
1556893992719,5cyclictest0-21swapper/110:15:251
1556893992719,4cyclictest0-21swapper/111:40:001
1556893992715,8cyclictest1600293-21grep09:10:241
1556888992720,3cyclictest0-21swapper/008:50:010
1556888992718,6cyclictest0-21swapper/009:20:010
1556901992620,3cyclictest1567867-21kthreadd07:40:003
1556901992620,3cyclictest1567867-21kthreadd07:40:003
1556901992620,3cyclictest0-21swapper/310:00:013
1556901992618,4cyclictest0-21swapper/309:20:273
1556901992615,6cyclictest1560343-21tail07:15:253
1556893992620,4cyclictest1629854-21kworker/u8:0+events_unbound11:07:391
1556893992617,6cyclictest0-21swapper/109:10:011
1556888992621,3cyclictest13550irq/42-30be0000.ethernet08:00:120
1556888992619,4cyclictest0-21swapper/009:35:000
1556888992619,4cyclictest0-21swapper/009:35:000
1556888992618,5cyclictest0-21swapper/010:25:240
1556888992616,7cyclictest1596365-21ntpq09:00:200
1556901992519,3cyclictest0-21swapper/311:10:253
1556901992518,3cyclictest0-21swapper/307:55:193
1556901992517,5cyclictest0-21swapper/311:50:003
1556901992517,5cyclictest0-21swapper/311:05:003
1556893992519,3cyclictest0-21swapper/108:45:231
1556893992518,4cyclictest0-21swapper/110:10:251
1556893992518,4cyclictest0-21swapper/109:15:271
1556893992516,6cyclictest0-21swapper/110:30:001
1556893992515,7cyclictest1576404-21sh08:05:011
1556893992512,4cyclictest1642434-21kworker/u8:2+events_unbound11:16:451
1556888992519,3cyclictest0-21swapper/010:45:250
1556888992518,4cyclictest13850irq/36-mmc109:55:010
1556888992514,8cyclictest1609930-21cut09:40:180
155688899251,3cyclictest1644442-21apt-get11:20:130
155688899251,3cyclictest1561079-21apt-get07:20:130
1556888992513,8cyclictest164-21systemd-journal11:44:200
155688899251,20cyclictest0-21swapper/011:01:150
1556901992416,4cyclictest0-21swapper/311:30:243
1556901992414,6cyclictest0-21swapper/308:20:013
1556893992417,5cyclictest1584974-21kworker/u8:0+events_unbound08:32:411
1556893992417,4cyclictest0-21swapper/111:25:241
1556893992417,3cyclictest0-21swapper/110:50:141
1556893992416,6cyclictest145-21jbd2/mmcblk1p2-809:20:101
1556893992413,7cyclictest164-21systemd-journal07:20:411
1556893992412,7cyclictest0-21swapper/111:45:271
1556888992418,3cyclictest0-21swapper/011:30:210
1556888992418,3cyclictest0-21swapper/009:55:150
155688899241,7cyclictest1581510-21sh08:20:010
1556888992417,4cyclictest1604336-21cron09:25:010
155688899241,3cyclictest1559358-21apt-get07:15:120
155688899241,19cyclictest0-21swapper/007:14:220
155688899240,20cyclictest0-21swapper/007:50:590
1556901992317,3cyclictest0-21swapper/312:05:233
1556901992316,3cyclictest0-21swapper/309:25:273
1556901992316,3cyclictest0-21swapper/308:53:513
1556901992314,6cyclictest0-21swapper/310:35:003
1556893992317,4cyclictest1562381-21kworker/u8:1+events_unbound07:38:071
1556893992317,4cyclictest1562381-21kworker/u8:1+events_unbound07:38:061
1556893992317,3cyclictest0-21swapper/107:25:151
1556893992311,9cyclictest1670008-21apt-get12:35:101
1556893992311,8cyclictest1661259-21ls12:10:001
1556888992317,3cyclictest13850irq/36-mmc109:30:010
1556888992316,3cyclictest0-21swapper/012:05:260
1556888992316,3cyclictest0-21swapper/011:12:190
1556888992313,6cyclictest0-21swapper/012:20:010
155688899231,18cyclictest0-21swapper/011:27:210
155688899231,18cyclictest0-21swapper/010:53:140
155688899231,18cyclictest0-21swapper/010:32:490
155688899231,18cyclictest0-21swapper/009:49:550
155688899230,19cyclictest0-21swapper/008:30:540
1556901992216,3cyclictest0-21swapper/312:25:203
1556901992215,4cyclictest0-21swapper/307:50:253
1556901992215,3cyclictest0-21swapper/310:50:153
1556901992214,6cyclictest0-21swapper/310:20:243
1556901992214,5cyclictest0-21swapper/308:00:163
1556893992216,4cyclictest1592077-21kworker/u8:2+events_unbound08:53:261
1556893992216,3cyclictest0-21swapper/109:55:151
1556893992216,3cyclictest0-21swapper/108:10:121
1556893992213,6cyclictest0-21swapper/110:05:121
1556888992218,2cyclictest0-21swapper/007:40:190
1556888992217,3cyclictest13550irq/42-30be0000.ethernet08:05:160
1556888992215,4cyclictest13550irq/42-30be0000.ethernet09:00:000
1556888992215,3cyclictest0-21swapper/008:52:150
155688899221,3cyclictest0-21swapper/008:40:170
155688899220,19cyclictest0-21swapper/010:43:010
155688899220,18cyclictest0-21swapper/010:07:500
155688899220,18cyclictest0-21swapper/007:33:040
155688899220,18cyclictest0-21swapper/007:33:030
1556901992114,4cyclictest1557616-21ntpq07:10:203
1556901992114,3cyclictest0-21swapper/312:34:433
1556901992113,5cyclictest0-21swapper/309:15:003
1556901992112,4cyclictest0-21swapper/309:05:253
1556901992111,7cyclictest1618160-21cut10:05:023
1556901992111,7cyclictest1578869-21ntpq08:10:203
1556893992116,3cyclictest1568755-21processes07:40:221
1556893992115,3cyclictest1582055-21cpuspeed08:20:121
1556893992115,3cyclictest0-21swapper/109:25:271
1556893992114,5cyclictest1666302-21kworker/u8:2+events_unbound12:30:231
1556893992114,4cyclictest1629707-21unixbench_singl10:35:251
1556893992114,3cyclictest1576886-21idleruntime08:05:171
1556888992117,2cyclictest0-21swapper/007:45:180
155690199209,8cyclictest1590795-21awk08:45:143
1556893992014,3cyclictest0-21swapper/110:45:231
1556893992013,5cyclictest231ktimers/107:10:121
1556893992012,5cyclictest16657772sleep112:20:251
1556901991912,5cyclictest391ktimers/312:10:113
1556901991911,5cyclictest391ktimers/310:15:133
1556901991911,3cyclictest0-21swapper/311:25:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional