You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-19 - 21:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Fri Jun 19, 2026 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1543115995844,9cyclictest311ktimers/211:00:112
1543118995243,5cyclictest1604632-21munin-plugin-st10:05:013
1543112994940,5cyclictest1594372-21cron09:35:001
1543112994737,7cyclictest1612703-21grep10:25:241
1543115994633,10cyclictest311ktimers/211:35:112
1543118994539,4cyclictest1565973-21kworker/u8:0+events_unbound08:38:433
1543112994538,4cyclictest1567761-21sendmail08:20:001
1543112994334,5cyclictest0-21swapper/107:45:011
154311899419,5cyclictest391ktimers/309:00:013
1543118994130,4cyclictest131rcu_preempt07:30:183
1543118994124,10cyclictest131rcu_preempt08:45:173
1543115994129,9cyclictest311ktimers/209:40:132
1543115994128,9cyclictest1565557-21grep08:10:262
1543118994034,4cyclictest1607978-21kworker/u8:2+events_unbound10:59:173
1543118994034,4cyclictest1599369-21kworker/u8:3+events_unbound09:54:023
1543118994034,4cyclictest1587084-21kworker/u8:3+events_unbound09:18:533
1543107994030,7cyclictest1597397-21grep09:40:260
1543107994015,14cyclictest0-21swapper/010:10:220
1543115993827,7cyclictest0-21swapper/211:55:252
1543112993826,8cyclictest1621656-21sh10:54:591
1543112993811,11cyclictest1617944-21seq10:41:191
1543112993811,11cyclictest1617944-21seq10:41:191
1543107993811,14cyclictest1563958-1kworker/0:2H+mmc_complete11:05:010
1543118993729,6cyclictest1583722-21cron09:05:013
1543118993729,5cyclictest1549177-21fschecks_time07:25:143
1543118993727,6cyclictest0-21swapper/309:15:003
1543118993726,7cyclictest0-21swapper/310:10:253
1543118993726,7cyclictest0-21swapper/308:20:273
1543118993726,7cyclictest0-21swapper/308:20:273
1543107993730,4cyclictest1577954-21unixbench_multi08:45:250
1543107993729,6cyclictest1648948-21unixbench_singl12:10:250
154310799370,34cyclictest0-21swapper/010:05:110
1543118993630,4cyclictest1523499-21kworker/u8:0+events_unbound07:13:273
1543118993630,4cyclictest1523499-21kworker/u8:0+events_unbound07:13:273
1543118993626,6cyclictest1648946-21grep12:10:253
1543118993626,6cyclictest0-21swapper/308:20:003
1543118993618,11cyclictest131rcu_preempt08:00:013
1543118993617,4cyclictest131rcu_preempt07:53:073
1543115993627,5cyclictest0-21swapper/212:20:152
1543115993627,4cyclictest0-21swapper/211:50:012
1543115993625,7cyclictest1595603-21grep09:35:242
1543115993625,7cyclictest1553407-21grep07:35:282
1543115993624,8cyclictest1629745-21grep11:15:262
154311299367,6cyclictest15822242cat09:00:011
154311299367,10cyclictest1609080-21sed10:15:241
1543112993629,4cyclictest0-21swapper/109:25:001
1543112993629,4cyclictest0-21swapper/109:25:001
1543112993628,5cyclictest1651091-21idleruntime-cro12:20:001
1543112993610,10cyclictest0-21swapper/112:23:471
1543118993528,5cyclictest1649050-21kworker/u8:1+events_unbound12:21:413
1543118993527,6cyclictest1602833-21kworker/u8:0+events_unbound10:17:423
1543118993527,4cyclictest0-21swapper/310:25:153
1543118993526,6cyclictest0-21swapper/311:55:003
1543118993525,6cyclictest0-21swapper/308:50:263
1543118993524,6cyclictest0-21swapper/312:29:273
1543118993515,10cyclictest131rcu_preempt11:40:153
1543118993510,17cyclictest131rcu_preempt10:20:323
1543115993526,6cyclictest1554670-21tr07:40:212
154311299359,9cyclictest1615071-21/usr/sbin/munin10:35:141
154311299357,11cyclictest1575281-21egrep08:40:141
1543112993526,6cyclictest1623359-21sh11:00:001
1543112993510,6cyclictest24-21ksoftirqd/108:45:231
1543107993519,6cyclictest131rcu_preempt10:18:560
1543118993427,4cyclictest0-21swapper/312:00:003
1543118993426,5cyclictest0-21swapper/311:50:013
1543118993422,4cyclictest131rcu_preempt11:00:123
1543118993419,5cyclictest131rcu_preempt07:40:143
1543118993419,4cyclictest131rcu_preempt08:40:263
1543118993418,5cyclictest131rcu_preempt10:40:003
1543118993417,6cyclictest1631754-21kworker/u8:2+events_unbound11:36:103
1543118993417,6cyclictest1617912-21kworker/u8:0+events_unbound11:27:063
1543118993417,5cyclictest131rcu_preempt12:05:253
1543118993415,10cyclictest0-21swapper/307:49:423
1543118993412,14cyclictest131rcu_preempt11:10:183
1543115993427,4cyclictest0-21swapper/207:25:242
1543115993424,6cyclictest0-21swapper/208:05:262
1543115993420,10cyclictest1638735-21grep11:40:242
1543112993426,4cyclictest0-21swapper/111:20:161
1543112993425,5cyclictest0-21swapper/111:00:251
1543112993423,7cyclictest1645508-21grep12:00:261
1543112993410,7cyclictest24-21ksoftirqd/110:00:011
1543107993427,4cyclictest0-21swapper/008:20:180
1543107993427,4cyclictest0-21swapper/008:20:180
1543107993427,3cyclictest0-21swapper/009:35:000
1543107993423,8cyclictest121ktimers/011:30:110
1543107993423,6cyclictest0-21swapper/011:40:010
154310799341,30cyclictest0-21swapper/011:15:120
154311899339,8cyclictest40-21ksoftirqd/307:25:003
1543118993323,6cyclictest0-21swapper/312:20:003
1543118993323,6cyclictest0-21swapper/311:20:253
1543118993323,6cyclictest0-21swapper/309:55:153
1543118993318,5cyclictest131rcu_preempt08:30:193
1543118993317,4cyclictest131rcu_preempt10:45:193
1543118993316,9cyclictest131rcu_preempt08:00:133
1543118993315,5cyclictest131rcu_preempt11:05:103
1543118993314,5cyclictest1601123-21kworker/u8:1-events_freezable_power_10:07:103
1543115993326,3cyclictest0-21swapper/211:20:252
1543115993323,7cyclictest311ktimers/210:35:112
1543107993326,4cyclictest0-21swapper/010:35:230
1543107993324,6cyclictest0-21swapper/009:20:010
1543107993316,10cyclictest131rcu_preempt08:30:510
154311899329,12cyclictest0-21swapper/309:20:233
154311899329,12cyclictest0-21swapper/309:20:223
1543118993224,4cyclictest0-21swapper/312:35:263
1543118993224,4cyclictest0-21swapper/312:30:243
1543118993223,6cyclictest0-21swapper/309:05:273
1543118993222,6cyclictest0-21swapper/308:05:263
1543118993221,4cyclictest131rcu_preempt10:40:143
1543118993221,4cyclictest131rcu_preempt10:40:143
1543118993217,4cyclictest131rcu_preempt07:15:203
1543118993217,4cyclictest131rcu_preempt07:15:193
1543118993213,9cyclictest131rcu_preempt12:00:183
1543118993213,9cyclictest131rcu_preempt10:50:183
1543118993213,10cyclictest131rcu_preempt09:33:003
1543118993212,9cyclictest131rcu_preempt09:36:493
1543118993212,10cyclictest0-21swapper/308:12:363
1543115993223,6cyclictest1602833-21kworker/u8:0+events_unbound10:14:072
1543115993222,7cyclictest1578402-21idleruntime-cro08:50:002
1543115993222,5cyclictest0-21swapper/212:30:242
1543115993221,7cyclictest1612684-21grep10:25:242
154311299327,11cyclictest0-21swapper/111:55:511
154311299324,10cyclictest1648466-21ntp_states12:10:211
1543112993225,4cyclictest1630209-21sendmail11:20:011
1543112993225,4cyclictest0-21swapper/111:10:151
1543112993224,4cyclictest0-21swapper/112:35:261
1543112993224,4cyclictest0-21swapper/107:15:271
1543112993224,4cyclictest0-21swapper/107:15:271
1543107993226,3cyclictest1626786-21kthreadd11:10:010
1543107993224,4cyclictest1581546-21grep08:55:250
1543107993223,5cyclictest0-21swapper/011:25:230
1543107993222,6cyclictest0-21swapper/007:35:280
1543107993211,11cyclictest131rcu_preempt12:31:310
1543118993121,7cyclictest381rcuc/309:48:473
1543118993121,5cyclictest0-21swapper/309:40:263
1543118993120,4cyclictest131rcu_preempt08:25:163
1543118993120,4cyclictest131rcu_preempt07:35:153
1543118993114,9cyclictest131rcu_preempt11:30:103
1543118993112,4cyclictest131rcu_preempt10:31:503
1543115993123,4cyclictest0-21swapper/207:10:172
1543115993123,4cyclictest0-21swapper/207:10:172
1543115993122,5cyclictest1590977-21sh09:25:002
1543115993122,5cyclictest1590977-21sh09:25:002
1543115993122,5cyclictest0-21swapper/210:20:262
1543115993121,7cyclictest0-21swapper/208:05:012
1543115993120,5cyclictest0-21swapper/210:20:002
1543112993123,4cyclictest0-21swapper/108:25:251
1543112993117,11cyclictest1654933-21munin-node12:30:101
154310799317,6cyclictest11-21ksoftirqd/007:46:230
1543107993116,9cyclictest131rcu_preempt12:25:370
1543107993116,8cyclictest131rcu_preempt07:53:100
1543107993116,6cyclictest131rcu_preempt10:41:410
1543107993116,6cyclictest131rcu_preempt10:41:410
1543118993016,4cyclictest131rcu_preempt11:15:153
1543115993022,4cyclictest0-21swapper/211:50:252
1543115993021,4cyclictest0-21swapper/208:20:172
1543115993021,4cyclictest0-21swapper/208:20:172
1543112993024,3cyclictest1639383-21apt-get11:45:121
1543112993023,4cyclictest1618493-21apt-get10:45:111
1543112993022,4cyclictest0-21swapper/107:35:171
1543112993019,7cyclictest1638080-21/usr/sbin/munin11:40:171
1543112993019,7cyclictest1591663-21/usr/sbin/munin09:25:201
1543112993018,9cyclictest1555941-21/usr/sbin/munin07:45:161
1543112993017,10cyclictest1582381-21/usr/sbin/munin09:00:171
1543112993016,11cyclictest1573155-21sshd08:33:411
1543112993013,13cyclictest1610529-21ntpq10:20:211
154310799309,8cyclictest1637777-21cat11:40:130
154310799307,6cyclictest11-21ksoftirqd/011:50:200
154310799306,5cyclictest11-21ksoftirqd/007:25:010
1543107993022,5cyclictest1652537-21grep12:20:250
1543107993021,6cyclictest0-21swapper/007:15:270
1543107993021,6cyclictest0-21swapper/007:15:270
1543107993020,7cyclictest1631457-21grep11:20:250
1543107993018,8cyclictest1587419-21sed09:15:010
1543107993018,4cyclictest131rcu_preempt08:42:080
1543107993016,3cyclictest131rcu_preempt10:25:120
1543107993016,10cyclictest121ktimers/008:00:120
1543107993015,5cyclictest131rcu_preempt07:58:340
1543118992913,5cyclictest131rcu_preempt09:25:163
1543115992917,8cyclictest1581582-21grep08:55:252
154311299293,9cyclictest629-21snmpd09:12:251
1543112992921,4cyclictest0-21swapper/107:53:341
1543112992919,7cyclictest1547748-21/usr/sbin/munin07:20:211
1543112992919,6cyclictest1626297-21grep11:05:241
1543112992918,8cyclictest1654608-21cron12:29:001
1543112992918,8cyclictest1588872-21cron09:18:011
1543112992918,7cyclictest1606894-21/usr/sbin/munin10:10:181
1543112992917,9cyclictest1578952-21/usr/sbin/munin08:50:151
1543112992917,9cyclictest1551471-21/usr/sbin/munin07:30:241
1543112992916,10cyclictest1647410-21cron12:07:001
1543112992916,10cyclictest1634046-21/usr/sbin/munin11:30:251
1543112992916,10cyclictest1565680-21cron08:11:001
1543112992916,10cyclictest1562413-21apt-get08:05:061
154310799295,5cyclictest11-21ksoftirqd/012:00:010
154310799294,9cyclictest11-21ksoftirqd/012:05:210
1543107992922,4cyclictest0-21swapper/011:00:000
1543107992921,5cyclictest1602476-21head09:55:250
1543107992914,8cyclictest131rcu_preempt07:44:140
1543107992912,8cyclictest131rcu_preempt09:22:400
1543107992912,8cyclictest131rcu_preempt09:22:400
1543115992820,4cyclictest0-21swapper/212:05:262
1543115992820,4cyclictest0-21swapper/211:00:002
1543115992820,4cyclictest0-21swapper/207:15:032
1543115992820,4cyclictest0-21swapper/207:15:032
1543115992818,5cyclictest0-21swapper/212:20:002
1543115992817,7cyclictest0-21swapper/209:05:272
154311299286,12cyclictest0-21swapper/111:39:141
1543112992820,4cyclictest0-21swapper/109:45:151
1543112992817,8cyclictest1601107-21cron09:54:011
1543112992817,8cyclictest1597456-21cron09:41:011
1543112992817,7cyclictest1574799-21cron08:38:001
1543112992816,9cyclictest1550204-21cron07:27:001
1543112992815,4cyclictest1543100-21cyclictest07:10:211
1543112992815,4cyclictest1543100-21cyclictest07:10:201
1543112992815,10cyclictest1595056-21meminfo09:35:171
154310799287,6cyclictest0-21swapper/008:50:160
154310799286,5cyclictest0-21swapper/007:30:150
154310799286,4cyclictest11-21ksoftirqd/008:05:140
154310799285,8cyclictest11-21ksoftirqd/012:35:220
154310799281,5cyclictest1585490-21latency_hist09:10:010
1543107992811,9cyclictest131rcu_preempt10:45:580
1543107992811,10cyclictest131rcu_preempt08:17:330
1543115992719,4cyclictest0-21swapper/209:10:262
1543115992717,7cyclictest1601196-21sh09:55:012
1543115992717,5cyclictest0-21swapper/211:29:212
1543112992720,4cyclictest0-21swapper/110:00:261
1543112992719,5cyclictest0-21swapper/108:25:001
1543112992719,5cyclictest0-21swapper/108:25:001
1543112992717,7cyclictest1642454-21cron11:54:001
1543112992716,7cyclictest1632458-21df_inode11:25:131
154310799279,9cyclictest131rcu_preempt07:10:180
154310799279,9cyclictest131rcu_preempt07:10:180
154310799278,9cyclictest131rcu_preempt10:52:310
154310799278,10cyclictest131rcu_preempt09:03:060
154310799276,7cyclictest11-21ksoftirqd/010:25:000
154310799275,5cyclictest0-21swapper/010:30:220
154310799272,7cyclictest11-21ksoftirqd/009:45:250
1543107992717,7cyclictest0-21swapper/011:15:010
1543107992710,8cyclictest131rcu_preempt08:13:400
1543115992620,3cyclictest0-21swapper/207:50:132
1543115992619,3cyclictest0-21swapper/212:25:142
1543115992617,6cyclictest1573348-21sort08:35:012
1543115992617,4cyclictest0-21swapper/208:50:162
1543115992614,7cyclictest1583738-21basename09:05:012
154311299264,13cyclictest24-21ksoftirqd/108:01:111
1543112992616,7cyclictest1614793-21sshd10:34:311
1543112992616,7cyclictest1604970-21/usr/sbin/munin10:05:141
1543112992616,7cyclictest1560302-21/usr/sbin/munin07:55:261
1543112992610,11cyclictest1584997-21grep09:05:271
154310799266,6cyclictest0-21swapper/010:00:170
154310799265,6cyclictest11-21ksoftirqd/007:30:010
154310799263,4cyclictest11-21ksoftirqd/009:30:010
1543107992617,6cyclictest121ktimers/012:00:260
1543107992612,6cyclictest131rcu_preempt09:51:300
1543107992612,4cyclictest131rcu_preempt11:45:110
154310799261,22cyclictest1574977-21sh08:40:000
1543115992515,6cyclictest0-21swapper/209:55:252
1543115992513,8cyclictest1-21systemd10:49:142
1543115992512,10cyclictest311ktimers/209:25:112
154310799259,9cyclictest131rcu_preempt08:28:050
1543107992517,5cyclictest0-21swapper/009:40:000
1543115992418,3cyclictest0-21swapper/210:00:112
1543115992416,4cyclictest0-21swapper/211:10:252
1543115992415,6cyclictest1644559-21sort12:00:122
1543115992415,4cyclictest0-21swapper/208:35:152
1543115992414,6cyclictest0-21swapper/210:05:252
1543115992412,8cyclictest1560716-21sendmail08:00:012
154310799243,5cyclictest11-21ksoftirqd/012:15:350
1543115992318,2cyclictest0-21swapper/207:45:132
1543115992317,3cyclictest0-21swapper/209:45:262
1543115992317,3cyclictest0-21swapper/209:15:162
1543115992317,3cyclictest0-21swapper/207:20:162
1543115992313,7cyclictest1616985-21cut10:40:162
1543115992313,7cyclictest1616985-21cut10:40:162
154311599231,12cyclictest0-21swapper/211:08:272
1543115992216,3cyclictest0-21swapper/210:50:242
1543115992216,2cyclictest0-21swapper/212:10:252
1543115992215,3cyclictest0-21swapper/212:35:252
1543115992213,6cyclictest311ktimers/209:30:202
1543115992115,4cyclictest1565973-21kworker/u8:0+events_unbound08:43:442
154311599209,8cyclictest1569987-21awk08:25:142
1543115991912,5cyclictest311ktimers/211:31:312
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional