You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-21 - 00:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sat Jun 20, 2026 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1373406995946,9cyclictest1539028-21sendmail12:40:010
1373408995731,23cyclictest1515599-21mailstats12:00:352
1373408995343,6cyclictest1409393-21idleruntime-cro09:10:012
1373406995340,5cyclictest0-21swapper/010:55:360
1373408995226,4cyclictest1420370-21unixbench_singl09:25:442
1373407995243,6cyclictest1475372-21ssh10:55:321
137340899515,13cyclictest428-21mta-sts-daemon07:30:282
1373406995134,4cyclictest0-21swapper/012:05:230
1373408995029,19cyclictest0-21swapper/209:20:012
137340699504,40cyclictest1524674-21sh12:15:300
1373408994939,6cyclictest1416042-21/usr/sbin/munin09:20:192
1373408994829,16cyclictest1003-21mta-sts-daemon10:06:572
137340899482,37cyclictest1432164-21rm09:45:282
1373408994823,5cyclictest1536759-21latency12:35:252
1373408994823,5cyclictest1412584-21latency_hist09:15:022
1373408994823,4cyclictest1447186-21fschecks_time10:10:232
1373408994823,2cyclictest0-21swapper/207:55:012
1373408994821,3cyclictest0-21swapper/211:57:442
1373408994821,3cyclictest0-21swapper/211:57:442
137340899481,25cyclictest1500018-21ssh11:35:312
137340899464,22cyclictest1524617-21ssh12:15:282
1373407994637,5cyclictest1471197-21cron10:50:011
137340899455,17cyclictest1462960-21ssh10:35:292
1373408994535,6cyclictest1466671-21unixbench_singl10:40:402
1373408994534,6cyclictest421-21dbus-daemon12:10:012
137340899453,35cyclictest1521516-21ssh12:10:282
1373408994524,4cyclictest1003-21mta-sts-daemon08:10:022
1373408994434,6cyclictest1428048-21unin-run09:40:012
1373407994234,5cyclictest1465086-21latency_hist10:40:021
137340799414,27cyclictest1490802-21ssh11:20:301
1373407994134,4cyclictest1431108-21awk09:45:011
1373407994133,5cyclictest1468149-21systemd-run10:45:011
137340799412,33cyclictest1410396-21ssh09:10:271
1373406994127,11cyclictest0-21swapper/007:40:010
1373408994029,8cyclictest421-21dbus-daemon11:30:022
1373408994026,10cyclictest1449926-21/usr/sbin/munin10:15:232
1373407994029,8cyclictest1498960-21munin-plugin-st11:35:011
1373408993930,6cyclictest1377922-21systemd-run07:25:012
1373408993927,9cyclictest1437610-21aten_r2power_cu09:55:182
1373407993928,8cyclictest1435292-21ntp_offset09:50:321
1373407993928,6cyclictest0-21swapper/110:10:361
137340699394,28cyclictest428-21mta-sts-daemon08:45:320
137340699392,32cyclictest12650irq/32-eth011:15:270
1373408993830,5cyclictest1505209-21latency_hist11:45:012
1373408993829,3cyclictest0-21swapper/211:20:412
1373407993831,4cyclictest1404874-21cron08:55:011
1373407993831,3cyclictest0-21swapper/112:15:281
1373407993830,5cyclictest1423472-21unixbench_singl09:30:441
1373407993830,3cyclictest0-21swapper/112:35:361
1373407993829,7cyclictest0-21swapper/109:45:181
1373407993828,5cyclictest1511884-21fschecks_time11:55:221
1373407993828,5cyclictest1511884-21fschecks_time11:55:221
1373406993828,7cyclictest561985-21cups-browsed11:40:290
1373408993730,5cyclictest1373405-21cyclictest12:20:282
1373408993725,9cyclictest1000-21mta-sts-daemon09:41:222
1373408993724,5cyclictest0-21swapper/211:05:032
1373407993730,4cyclictest1529767-21ssh12:25:011
1373407993729,5cyclictest1400402-21sendmail08:40:011
1373407993728,5cyclictest0-21swapper/112:30:261
1373406993722,12cyclictest0-21swapper/010:10:010
1373406993718,12cyclictest1429058-21ntpq09:40:310
137340899363,26cyclictest1506221-21ssh11:45:302
1373408993625,7cyclictest420-21avahi-daemon10:27:152
1373406993630,3cyclictest0-21swapper/011:35:370
1373406993616,2cyclictest171rcu_preempt10:47:590
137340699361,5cyclictest171rcu_preempt11:00:290
1373408993528,4cyclictest1441972-21unixbench_multi10:00:412
1373408993528,4cyclictest1002-21mta-sts-daemon11:15:362
1373408993528,3cyclictest0-21swapper/211:30:342
1373408993526,6cyclictest420-21avahi-daemon08:35:232
1373408993526,4cyclictest0-21swapper/209:30:232
137340899352,26cyclictest561985-21cups-browsed10:30:332
137340799359,17cyclictest1438373-21ssh09:55:311
1373406993528,5cyclictest1483570-21systemd-run11:10:010
1373406993528,4cyclictest1453377-21/usr/sbin/munin10:20:220
1373406993527,6cyclictest1417182-21unixbench_multi09:20:410
1373408993426,5cyclictest1435807-21unixbench_singl09:50:402
1373407993428,4cyclictest1428007-21awk09:40:011
1373407993427,4cyclictest1395357-21ntp_offset08:20:301
1373407993427,4cyclictest0-21swapper/107:55:011
1373407993426,5cyclictest1483669-21latency_hist11:10:021
1373407993425,6cyclictest1532882-21unin-run12:30:011
1373407993424,6cyclictest420-21avahi-daemon10:20:171
1373406993424,8cyclictest1463290-21switchtime10:35:390
1373408993325,5cyclictest1471207-21idleruntime-cro10:50:012
1373408993324,6cyclictest1001-21mta-sts-daemon11:12:342
1373408993318,10cyclictest1001-21mta-sts-daemon07:10:362
137340799333,23cyclictest1416746-21ssh09:20:331
1373407993326,3cyclictest0-21swapper/111:40:011
1373407993325,5cyclictest1505248-21latency_hist11:45:021
1373407993325,5cyclictest1443474-21systemd-run10:05:011
1373407993324,5cyclictest1508773-21sort11:50:221
1373407993322,8cyclictest1408740-21ntp_kernel_err09:05:291
1373407993319,10cyclictest421-21dbus-daemon11:18:161
137340699337,19cyclictest12650irq/32-eth010:30:280
137340699337,19cyclictest0-21swapper/011:28:440
137340699334,21cyclictest1413502-21ntpq09:15:310
1373406993327,4cyclictest1391241-21unixbench-2d08:05:400
1373406993326,4cyclictest1409392-21munin-plugin-st09:10:000
1373406993324,7cyclictest1400436-21latency_hist08:40:020
1373406993324,4cyclictest0-21swapper/012:25:260
1373406993314,5cyclictest1484531-21ssh11:10:270
1373408993226,3cyclictest0-21swapper/210:55:012
1373408993225,4cyclictest1397919-21fschecks_time08:30:212
137340899321,4cyclictest171rcu_preempt07:55:302
1373407993224,4cyclictest0-21swapper/107:35:331
1373407993224,3cyclictest0-21swapper/112:05:221
1373407993223,5cyclictest0-21swapper/108:35:011
137340799321,24cyclictest1493966-21ssh11:25:341
1373406993225,5cyclictest420-21avahi-daemon07:20:350
1373406993224,5cyclictest1423015-21proc_pri09:30:380
1373406993222,8cyclictest1386777-21unixbench_multi07:50:390
137340699322,26cyclictest1466073-21ssh10:40:280
1373406993219,8cyclictest12650irq/32-eth011:30:390
1373406993219,8cyclictest12650irq/32-eth009:45:330
1373408993124,4cyclictest1000-21mta-sts-daemon08:40:212
137340799317,13cyclictest1419879-21ssh09:25:381
1373407993125,3cyclictest1383923-21kworker/u8:1507:45:011
1373407993125,3cyclictest0-21swapper/111:45:161
1373407993124,3cyclictest0-21swapper/110:05:341
1373407993123,5cyclictest1413137-21fschecks_count09:15:251
1373407993122,6cyclictest1523803-21latency_hist12:15:021
137340699315,5cyclictest0-21swapper/008:50:330
1373406993125,2cyclictest0-21swapper/010:15:200
1373406993124,5cyclictest566-21gmain10:50:270
1373406993124,5cyclictest191rcuc/011:55:340
1373406993124,5cyclictest191rcuc/011:55:340
1373406993122,7cyclictest1003-21mta-sts-daemon08:40:350
137340899303,21cyclictest1533457-21ssh12:30:202
1373408993025,2cyclictest0-21swapper/207:40:012
1373408993022,4cyclictest0-21swapper/210:20:352
1373408993019,7cyclictest561985-21cups-browsed10:59:432
1373407993024,4cyclictest1373405-21cyclictest07:45:321
1373407993023,4cyclictest1451207-21unixbench_multi10:15:411
137340699309,7cyclictest151ktimers/009:10:270
1373406993025,3cyclictest1389905-21grep08:05:010
1373406993024,4cyclictest1425324-21aten_r2power_fr09:35:190
1373406993021,6cyclictest1385395-21munin-plugin-st07:50:020
137340699301,24cyclictest1419855-21sshd09:25:360
1373406993010,16cyclictest1506098-21ntpq11:45:250
137340899294,13cyclictest1530786-21ssh12:25:272
1373408992922,3cyclictest0-21swapper/208:50:262
1373408992922,3cyclictest0-21swapper/208:20:002
1373408992922,3cyclictest0-21swapper/208:20:002
137340799294,12cyclictest1409395-21diskmemload10:28:131
1373407992920,6cyclictest1393416-21fschecks_time08:15:211
1373407992920,6cyclictest1393416-21fschecks_time08:15:211
1373407992920,4cyclictest0-21swapper/110:53:431
137340699296,8cyclictest0-21swapper/007:43:220
1373406992923,4cyclictest1529978-21latency_hist12:25:020
1373406992922,4cyclictest1491812-21ssh11:21:560
1373406992921,3cyclictest14-21ksoftirqd/007:10:280
1373406992915,11cyclictest191rcuc/008:55:300
137340899282,17cyclictest1402796-21modprobe08:45:292
1373408992820,4cyclictest1000-21mta-sts-daemon11:05:302
1373408992813,10cyclictest421-21dbus-daemon07:45:282
137340799282,22cyclictest1459914-21ssh10:30:301
1373407992821,3cyclictest0-21swapper/111:11:501
1373407992820,5cyclictest1478022-21fschecks_time11:00:221
1373407992815,8cyclictest171rcu_preempt08:00:301
137340799281,4cyclictest1405865-21ntpq08:55:351
137340699282,3cyclictest171rcu_preempt10:25:310
1373406992822,4cyclictest1437727-21aten_r2power_po09:55:180
1373406992818,5cyclictest14-21ksoftirqd/012:03:140
1373406992816,5cyclictest1394020-21sensors_temp08:15:370
1373406992816,5cyclictest1394020-21sensors_temp08:15:370
137340699281,24cyclictest1441457-21ssh10:00:320
137340699280,5cyclictest171rcu_preempt11:50:350
1373408992721,4cyclictest1373405-21cyclictest07:40:312
1373408992718,6cyclictest561985-21cups-browsed09:00:312
137340899271,21cyclictest421-21dbus-daemon11:53:422
1373407992723,2cyclictest0-21swapper/108:15:011
1373407992723,2cyclictest0-21swapper/108:15:011
1373407992720,3cyclictest0-21swapper/112:00:331
1373407992717,5cyclictest171rcu_preempt08:25:201
1373407992717,3cyclictest171rcu_preempt08:00:011
1373406992722,3cyclictest151ktimers/012:10:300
1373406992719,6cyclictest1533610-21kernelversion12:30:230
1373406992718,4cyclictest171rcu_preempt07:55:230
1373406992716,3cyclictest171rcu_preempt08:30:400
1373408992618,5cyclictest1395901-21cron08:25:002
1373407992619,5cyclictest1379297-21unixbench_multi07:25:401
1373407992618,5cyclictest1407890-21cron09:05:011
1373407992618,5cyclictest1377822-21unixbench_singl07:20:401
1373407992618,2cyclictest1401924-21systemd08:45:021
1373407992616,7cyclictest1403389-21/usr/sbin/munin08:50:021
1373406992618,6cyclictest1375424-21sshd07:15:230
1373408992519,3cyclictest0-21swapper/207:15:412
137340899251,19cyclictest1406384-21awk09:00:012
1373406992520,2cyclictest0-21swapper/008:10:230
1373406992520,2cyclictest0-21swapper/008:10:230
1373406992519,4cyclictest1407836-21grep09:00:400
1373406992519,4cyclictest1380843-21unixbench_singl07:30:400
1373406992517,6cyclictest1434633-21aten_r2power_po09:50:190
1373406992514,3cyclictest171rcu_preempt10:10:190
1373406992513,9cyclictest1378869-21ntpq07:25:330
137340899244,13cyclictest428-21mta-sts-daemon07:25:282
1373408992418,3cyclictest1003-21mta-sts-daemon08:14:172
1373408992418,3cyclictest1003-21mta-sts-daemon08:14:172
137340799245,13cyclictest428-21mta-sts-daemon07:15:281
1373407992419,2cyclictest0-21swapper/107:35:011
1373406992417,5cyclictest1373418-21systemd-run07:10:010
1373406992417,5cyclictest1373418-21systemd-run07:10:010
1373406992417,5cyclictest0-21swapper/008:25:240
1373408992317,2cyclictest0-21swapper/208:25:232
1373407992315,6cyclictest1000-21mta-sts-daemon08:05:311
1373406992318,2cyclictest0-21swapper/008:25:000
1373407992210,3cyclictest171rcu_preempt07:10:221
1373408992116,2cyclictest0-21swapper/208:00:202
1373407992113,4cyclictest0-21swapper/107:10:011
1373407992113,4cyclictest0-21swapper/107:10:011
137340999183,6cyclictest0-21swapper/311:20:303
137340999182,8cyclictest0-21swapper/308:30:273
137340999182,7cyclictest0-21swapper/307:20:323
137340999182,5cyclictest0-21swapper/307:10:343
137340999172,6cyclictest0-21swapper/307:40:333
137340999171,9cyclictest0-21swapper/312:10:293
1373408991711,4cyclictest0-21swapper/207:10:012
1373408991711,4cyclictest0-21swapper/207:10:012
137340999164,7cyclictest0-21swapper/309:55:213
137340999162,8cyclictest0-21swapper/309:10:283
137340999161,8cyclictest0-21swapper/310:40:283
137340999161,8cyclictest0-21swapper/309:35:273
137340999161,7cyclictest0-21swapper/310:50:323
137340999161,7cyclictest0-21swapper/310:15:303
137340999161,7cyclictest0-21swapper/309:30:383
137340999161,6cyclictest0-21swapper/308:55:283
137340999154,5cyclictest0-21swapper/310:25:313
137340999152,9cyclictest0-21swapper/311:55:393
137340999152,9cyclictest0-21swapper/311:55:393
137340999152,7cyclictest0-21swapper/312:05:233
137340999151,8cyclictest0-21swapper/311:15:263
137340999151,7cyclictest0-21swapper/310:55:293
137340999151,6cyclictest0-21swapper/309:45:273
137340999143,5cyclictest0-21swapper/311:30:243
137340999142,8cyclictest0-21swapper/311:35:283
137340999142,8cyclictest0-21swapper/311:05:353
137340999142,8cyclictest0-21swapper/310:20:243
137340999142,7cyclictest0-21swapper/311:45:303
137340999142,7cyclictest0-21swapper/311:10:293
137340999142,6cyclictest0-21swapper/307:45:283
137340999142,6cyclictest0-21swapper/307:25:323
137340999142,6cyclictest0-21swapper/307:15:303
137340999141,8cyclictest0-21swapper/312:30:283
137340999141,8cyclictest0-21swapper/311:40:233
137340999141,8cyclictest0-21swapper/310:35:033
137340999141,7cyclictest0-21swapper/312:35:233
137340999141,7cyclictest0-21swapper/312:20:303
137340999141,7cyclictest0-21swapper/310:10:343
137340999141,7cyclictest0-21swapper/309:25:333
137340999141,6cyclictest0-21swapper/308:05:283
137340999141,6cyclictest0-21swapper/307:50:263
137340999140,6cyclictest0-21swapper/308:25:303
137340999140,6cyclictest0-21swapper/308:00:353
137340999135,5cyclictest0-21swapper/308:35:293
137340999134,6cyclictest0-21swapper/308:20:293
137340999133,6cyclictest0-21swapper/309:15:393
137340999133,5cyclictest0-21swapper/311:00:303
137340999132,6cyclictest0-21swapper/309:50:223
137340999132,6cyclictest0-21swapper/309:20:213
137340999132,6cyclictest0-21swapper/308:40:323
137340999132,5cyclictest0-21swapper/309:00:283
137340999131,7cyclictest0-21swapper/311:25:273
137340999131,7cyclictest0-21swapper/310:45:283
137340999131,6cyclictest0-21swapper/309:05:303
137340999131,6cyclictest0-21swapper/308:50:333
137340999131,6cyclictest0-21swapper/308:15:293
137340999131,6cyclictest0-21swapper/308:15:293
137340999125,5cyclictest0-21swapper/310:35:353
137340999123,6cyclictest0-21swapper/312:25:183
137340999123,6cyclictest0-21swapper/312:05:003
137340999123,6cyclictest0-21swapper/309:40:213
137340999122,6cyclictest0-21swapper/311:50:223
137340999121,7cyclictest0-21swapper/312:15:253
137340999121,7cyclictest0-21swapper/310:05:233
137340999121,7cyclictest0-21swapper/308:45:293
137340999121,6cyclictest0-21swapper/308:10:293
137340999121,6cyclictest0-21swapper/308:10:293
137340999121,6cyclictest0-21swapper/307:55:193
137340999121,6cyclictest0-21swapper/307:35:333
137340999121,6cyclictest0-21swapper/307:30:343
137340999112,6cyclictest0-21swapper/310:00:173
13734099982,5cyclictest0-21swapper/307:10:013
13734099982,5cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional